How to Apply A/B Testing to Your Emails

OstapWritten by OstapJun 22, 2016

Every internet marketer knows the secrets of successful online campaign: know your audience, use best practices and test, test, test. A/B testing is crucial when it comes to newsletters. How to apply it properly? Here are our simple pieces of advice, based on personal experience.

Start with software

Whatever you’re doing with your newsletters, start with picking up the software. The majority of modern online marketing tools allow an option for A/B testing. For example, popular MailChimp asks about that in the beginning, offering you a few types of a campaign (regular, plain-text, A/B testing and others).

Therefore, define the automation which will make the job for you. It should take your entire list of subscribers, split them into two groups and send letters to each of them. Better to test at the same time, on the list of the same interest. Another way, you will not get accurate data, as it will be affected by time difference and specific features of a target group.

Never test more than 3 options in A/B testing

Obviously, not for everyone. Many people run chaotic campaigns, test a lot of features and then cannot manage the results. A/B testing need just two options, as its name tells. However, as an exception, you can try three. For instance, if you test call to action buttons or other small things.

Define what you test

Yes, here your testing starts. Define what you test.

Here are a few hints what to test:

  • Subject line.
  • Sender (person or company).
  • Call to action.
  • Plain text or design email.
  • Short letter with one element or many elements combined.
  • Images.
  • Content with questions, specific words or triggers.
  • Any other pieces of text.
  • Time of sending.

This options do not limit your imagination. Depending on your business, you test different parts of your letter. In particular, online retailer can add or miss the price of goods. Think over your need.

Chose measurable elements

Once you chose a feature for A/B testing, come through it and check if you can get clear result which are applicable for next messages. For example, if you test call to action buttons, one will perform better than another – and you can will use it in future.

However, when you use different descriptions on the same topic, how can you understand the cause of better performing letter? Better to define one feature, like expressive word and more common one, or beginning with and without a name, or question sentence, or discount before a call to action.

Always test pieces which can them make a rule for your future campaigns.

Summarize the result of A/B testing

Create simple document to insert the results of every campaign and lessons learnt.

Starting your next newsletter, you can simply open it and use as a checklist: if your subject line fits the rule from previous A/B testing, if CTA is one better performing, and so on.

When you have results, remember to use them.

Take into account all metrics

Depending on what you test, check the proper metrics which define the result. For the subject line, it’s open rate. For content – it’s click rate, bounce rate, unsubscribing.

However, keep in mind to check all metrics. Thus, once we tested two buttons, typical “Shop now” and something like “My home needs PRODUCT NAME”. Click rate and conversion were almost the same. However, with a direct call to buy much more people unsubscribed from our news. Therefore, we concluded to use soft CTAs.

Here are simple pieces of advice on your way to perfect A/B testing campaigns. Drop us a line which tips or techniques you use. We at Newoldstamp are already happy to hear from you.


Ostap is a strong marketing professional with 3+ years of experience in the software industry. Skilled in Marketing Strategy, Digital Marketing, SMM, Content marketing, Search Engine Optimization (SEO), Google Analytics and Google Adwords.

Go to Ostap’s Profile

Create Signatures for Your Team

Sign up