The Ultimate Guide to A/B Testing: Email Marketing for Ecommerce


In today’s digital age, online retailers are using a huge variety of tools to market their business. Email marketing remains the most powerful tactic for reaching potential customers, especially when paired with good A/B testing, you can fine-tune your messages to drive substantial conversion results.

There are multiple ways to test the efficacy of your email campaigns. You can experiment with different variants of your email’s structure and switch it up.

You can test emails in order to see what your recipients respond by using different Subject Lines, different layouts and optimize it’s open and click-through rates.

What Is A/B Testing?

A/B testing, which may also be referred to as split or bucket testing, is a method in which marketers compare two versions of an email (or web page, app, etc.) to see which one performs better. In an A/B test, more than two variants of an email are different, and they’re shown to users at random.

When using A/B tests as part of your strategy, you create one version of your email and this will serve as your primary email. Then, you need to create a “B” version of this email, either by editing the original or altering it altogether, tweaking things as small as subject wording, the greeting or something significant as the entire layout and design of the email.

How A/B Testing Works

In order to benefit from A/B testing’s full potential, marketers should create a test list in order to control the results of their testing. ContactPigeon’s campaign manager can support the A/B testing process.

The tool will randomly select a pre-specified percentage of your contacts to send a specific email template.

For example, the first 15% of the contacts are randomly selected for the control email, while another 15% will receive the test variant.

Based on how your contacts reacted by opening or clicking, there will be a winning version and that will be sent to the rest of your contact list.

With this type of A/B testing tool, marketers can then measure and analyze how users are interacting with both emails and how this is preventing your visitor’s engagement and more conversions.

Tips for A/B Testing Email Marketing

If you’re looking into using A/B testing as part of your email marketing strategy for your company, the following tips can guarantee you the results you’re looking for:

Know What You’re Testing (and Why You’re Testing it)

One of the key steps to take before beginning an A/B test is to figure out what you’re testing and why. Choosing variants at random is not an effective approach, you should always think about what problems you are encountering with your marketing emails: are people not opening them? If so, consider testing your subject line.

Are they not clicking the links inside? If that’s the case, you may want to think about testing your CTAs. Pick one objective that you would like to improve on, and make minor improvements in the variant version to test for outcomes.

It’s important to improve results gradually by taking on one improvement at a time during A/B testing. Too many changes often confuse the results as you are introducing multiple elements that may impact the outcomes positively or negatively at the same time.

Focus on one objective and work on optimizing the result just for that.

Email Characteristics to Test

If you’re not sure what you want to test in your A/B test, here are several key characteristics you can analyze to help you increase conversions.

  • Open Rate: If you send subscribers an email, you want them to open it. You can use your A/B test to figure out how to maximize your open rate, which is the number of emails that are opened divided by the difference between the number of emails that were sent and the number of emails that bounced or were undeliverable.
  • Click Rate: When you include a link in your email to take users to your site (or a landing page, etc.), you want to make sure that they click it. In your A/B test, you can tweak factors to see what helps to increase the number of subscribers that choose to click on links they see in the body of the email.
  • Conversion Rate: The ultimate goal of a marketing email is to convince a customer to make a purchase. You can also use an A/B test to optimize your conversion rate — that is, the number of people that actually buy once they’ve opened your email and clicked the link(s) inside of it.

A/B Email Testing Ideas

There is a variety of things you can test when conducting an A/B email test. However, the following list contains some of the most valuable tests you should consider for A/B testing.

  • Subject line: Tweak your subject line to help test your open rate. Consider slight changes in wording, like saying “Sale on favorite items” vs” “Your favorite items are discounted now.”
  • Call to Action: Change up the wording and look of the CTA in your emails. Try doing things like using a more imperative CTA (Click here to buy!) vs. a more straightforward suggestion (Find out More Details Here)
  • Design: Flashy design can work in marketing emails, but sometimes, a plain text email is simpler for consumers. Vary your email designs in the two emails — one with a flashier, more complex look, and one with simple text.
  • Time-of-Day: Consider testing if the time of day matters when sending your emails. Send one group of emails in the morning before work, and one during the lunch hour, when people are at work. See which performs better.
  • Personalization: Try personalizing emails to one group in your test and leaving the other group of emails not personalized. See if addressing consumers directly in places like the subject line can convince them further to open, click and buy.

We hope you take into account these steps above as it will help your decision-making process and of course will bring you priceless results!

Loved this article? We also suggest:


View posts by ContactPigeon
ContactPigeon is the ultimate analytics and marketing automation platform. We provide a solution to every marketer's problem: How to send the right message, at the right time.