A Step-by-Step Guide to Creating the First AB Test for Your E-Commerce Store

A Step-by-Step Guide to Creating the First A/B Test for Your E-Commerce Store

Running an e-commerce store is difficult especially when you are successful in driving a significant amount of traffic, but not able to see conversions proportional to the traffic. You can’t expect the situation to change by doing things the same way. As Albert Einstein said, “The definition of insanity is doing the same thing over and over again and expecting a different result.”

To get better results, you need to experiment. A/B testing is a potent methodology in the arsenal of marketers when it comes to optimizing the conversion rate for e-commerce stores.

A/B testing (Also known as split testing) is a method of creating two variations of a web page to measure which version performs better.

In this article, we will look at how you can create an A/B test for your e-commerce store.

Which Web Elements Should You A/B Test?

The purpose of A/B testing is to optimize elements on your webpage that either eliminate the potential obstacles visitors might be facing during their shopping process or add elements that would speed up the buying process. Here is a list of web elements that can be A/B tested:

  1. Page layout
  2. Value proposition
  3. Headlines
  4. Sub-headlines
  5. Body copy
  6. Product description
  7. Rich media: Product images, demo videos, etc.
  8. Call-to-action text
  9. Call-to-action button: Size, shape, and color
  10. Social proof and security seals: Testimonials, reviews and ratings, trust badges
  11. Pricing options: Discounts, scarcity, EMI options, offers, etc.
  12. Checkout: Form field, single vs. multi-step checkout process, etc.
  13. And any element you think is worth testing

Caveats Before Running an A/B Test

Before we look at the actual process to set up an A/B test, let’s look at 3 main caveats (or mindsets if you will) you need to be aware of:

1. Test Only One Variable at a Time

To observe a significant impact of a test, make sure to test only one variable at a time. So, if you are experimenting with CTAs, stick exclusively to them.

Also, when starting out it is strongly recommended not to run multiple tests simultaneously. Running various tests or testing multiple variables parallelly, might get you results, but you won’t be able to gauge the impact of individual tests and variables.

2. Realize That A/B Testing Is a Process

If you are looking at A/B testing as a short-term way to increase your revenue, it will rarely work for you. A/B testing is an iterative process where you need to be patient in the early stages before you start seeing significant results in your conversions.

3. Make Sure You’ve Enough Traffic on Your Website

If you are not getting enough traffic on your website, it will be difficult to split your traffic to run tests and even if you do so, the results you’d get won’t be of much value. Therefore, if you don’t have enough traffic at the moment, try to increase it through SEO, paid media and social media activities before running your first A/B test.

Steps to Create an A/B Test for Your E-Commerce Store

Step 1: Define Your Objectives

Let’s start at the beginning. Before defining your objective for running an A/B test, have a look at your current data. Go through your analytics software, heatmaps, visitor session recordings and surveys to gather website data and customer behavior and identify problem areas. You’d find problems such as high bounce rate, certain areas on the website getting no attention, people leaving your website during a particular stage of checkout etc.

Based on the problems identified, pick one area and create an objective around it. The objective should not be a vague statement such as decrease cart abandonment. Instead, make it concrete by assigning a number to it such as decrease cart abandonment rate by 10%. A clearly defined goal will help you gauge whether you were able to achieve your objective or not.

Step 2: Develop a Hypothesis

Based on your objective, come up with a hypothesis that would help you boost your conversions. For example, your objective could be to reduce drop-offs from the product page. So you can come up with a list of hypotheses such as:

  1. Introduce a demo video
  2. Use action words in the product headline
  3. Reduce the product description copy
  4. Tweak the call-to-action button

It’s a good idea to make prominent changes on your pages in the early stage of your A/B tests. Doing so will help you see the direct impact rather than beating around the bush to see minute improvements.

Now, based on the Analytics data and heatmaps, you might notice that people are rarely clicking on your Add to Cart CTA. Here, you have three options:

  1. Either tweak the copy of the CTA
  2. Change the color of the CTA button
  3. Increase the size of the CTA button


Based on the above suggestion, we go with the third option and develop a hypothesis as follows: Increasing the size of the CTA button will increase the overall conversions.

Step 3: Design Your A/B Test

It’s time to put your assumption to the test. Pick a testing platform like HumCommerce to run your A/B test. You need to create a new version (challenger) of the original page consisting of your hypothesis to test your assumption. You do not require technical competence to make changes in your challenger page. You can make the changes through the inbuilt WYSIWYG (What You See Is What You Get) editor in HumCommerce.

In the beginning, direct only 15-20% of the traffic to the challenger version as it is not certain how well your visitors will receive it. You will still get accurate results since you will measure the conversion rate and not just conversions. As the challenger version starts gaining good traction, you can direct more and more traffic to it.

Step 4: Launch the Test

Once you launch the A/B test, let it run for a significant amount of time to gather actionable data. Do not stop or pause the test as soon as you start seeing a clear winner. Be very patient during this step — the consumer behavior changes based on the nature of the campaign, traffic source, day and time of the week, location, device, etc. Therefore, even though you might be tempted to pick a winner, run your test for at least 3-4 weeks before reaching to a conclusion.

Step 5: Collect and Analyze Data

Collect and analyze data when the test is complete. Measure how each variant performed. Did the challenger version outperform the control version? If your assumption was correct, go ahead with its implementation, or else go through another iteration of the process. Remember the second caveat. A/B testing is a process, and you need to keep iterating until you get the desired results.

Closing Words

Many marketers give up completely on A/B testing after their first few tests fail. The failure might happen due to a multitude of reasons. In such cases, you should go through your Analytics and CRO tool thoroughly to understand the meat of the matter instead of abandoning it altogether.

Developing a culture of A/B tests will over time help you reduce cart abandonment and customer acquisition cost, and increase average order size and customer lifetime value.

Editorial Staff

The editorial staff takes care of growth strategies, content creation, and outreach within the community for HumCommerce.