SMS content testing at Premera Blue Cross

premera-text-messages

Product at-a-glance

Premera Blue Cross launched it's first mobile messaging platform on December 1, 2017. From that date, members can sign up to receive SMS related to their health plan.

IMPACT

Thanks to testing with participants, and working collaboratively with our content strategist, we were able to:

  • Use data to convince stakeholders that each text message should have one and only one call-to-action.

  • Improve clarity of the text messages

  • Change the cadence of the text messages

ROLES AND DELIVERABLES

  • Conducted iterative testing on individual text messages to improve clarity

  • Implemented first brand attribute system to evaluate our messages against

  • An ever-expanding list of SMS best practices

TOOLS

  • SurveyMonkey

  • Sketch


Iterative content testing

sms-testing-welcome.png

Defining the one core message
160 characters, the length of an SMS, is not a lot to work with. Many leaders at Premera saw the new text messaging service as a chance to get members to do nearly everything at once: 

  1. Confirm member has signed up for text messages

  2. Ask them to sign up for an online Premera account)

  3. Include required legalese

We A/B tested the messages with SurveyMonkey. In our original draft, we found that trying do do multiple things at once confused users. The shortened link looked like spam, and trimming character count by removing letters from words (message & data to msg&data) decreased confidence.

 

Implementing baseline brand attributes

sms-attributes

How do you know that an SMS is (1) good enough to be sent out to customers, and (2) provides actual value?

With every text message we A/B test, we include a short matrix rating scale for participants to rate how friendly, helpful, clear, and secure the message is. This helps our content team evaluate how effectively our messages are written, and provides hard data to show where our copy is succeeding or failing.

 

Mini quiz: do people know what the CTA does?

premera-sms-nurseline.png

On the right is a text message advocating for a free service for Premera members called NurseLine.

When including the link for the original draft (left), we asked "if you tapped on the link, what do you believe would happen next?" Technically, the correct answer was that it would open up a contact card that the person could easily add to their phone. However, less than 10% of people believed that would happen - the majority reasonably believed that it would open a webpage or a NurseLine app to sign up for.

Based on these findings, the content strategist and I revised the SMS to make it clear that you could save the phone number with NurseLine directly to your contacts.

 

Launch & best practice learnings

laurie-release-day

On the right, see a launch day picture of my content strategist and right-hand woman Laurie who worked tirelessly with me to write and test our SMS content.

Together, we have identified an ever-expanding list of SMS best practices:

  • One text message, one call-to action (CTA)

  • Make sure the CTA is directly related to value that the customer will receive

  • Try to put the customer value up front. (For example "Free 24-Hour NurseLine is included in your Premera plan" puts the value right at the beginning of the message.)

  • Use full-spelled-out words and correct punctuation. Failing to do so erodes trust.

  • Be as clear as humanly possible in 160 characters. Try to address possible ambiguities up front...but you may find more after testing your content!

  • Don't use links - they decrease confidence in the sender (is this from my health insurance company, or from a spammer?) and eat up valuable characters

  • Include the company name in each SMS

  • Always include required legalese