Beginners guide to AB testing

Beginners guide to A/B testing for Ecommerce

What do you do when visitors leave your store without buying?

You have two options:

  1. Drive more traffic to your store, or
  2. Optimize your store so that more visitors buy

Driving traffic is expensive compared to optimizing your store. And if your store is not optimized, there’s no guarantee that increased traffic will lead to an increase in sales.

Tools like the HumCommerce heatmaps and visitor recordings can help you pinpoint how visitors interact with your store. You can find the exact problem why visitors leave your store.

Once you recognize the problem, you may be tempted to fix it. But how do you know if your fix will work? Or make things even worse?

The answer is A/B testing.

“Every aspect of marketing is entirely useless unless it produces conversion.”
Jeremy Smith

What is A/B Testing?

AB testing

A/B testing is also known as Split Testing or Bucket Testing. It is an experiment that is used to compare two different versions, A and B, of a single variable to find out which one is more effective simply.

It’s a website optimization method that lets you compare two different versions of a web page by using live traffic.

A/B testing can involve tracking,

  • how visitors are interacting with pages,
  • whether they are watching videos or not,
  • if or not they are signing up, what are the buttons they are clicking upon, etc.?

You can quickly assess the most effective page out of the two options.

It’s different from multivariate testing

People often compare this methodology with Multivariate testing.

Although both of them carry a similar fundamental system; however, they’re different.

Multivariate provides more comparing variables and even provides additional information regarding the interaction of variables.

How to Do A/B Testing?

Initially, A/B test may seem like a straightforward procedure and a plain sailing. You may think, creating two variants of a webpage with different button colors may do the trick. You’ll get the data as to what color brought the maximum conversion, and that’s it.

“You start by figuring out which pages cause the biggest drop-offs – where the flow is stuck. Once you understand WHERE the problem is, you proceed to identify WHAT the problem is.”
― Peep Laja

So, how to do A/B testing that brings valuable results? Here is everything that you should know about it:

Zeroing Upon the Business Goals

You must begin by defining the goals of business websites and perusing the performance indicators. This is the stage that might fill up your mind with unnecessary questions. Hence, it might become a bit difficult to crack down integral answers, such as: What is the goal of your website?

At this point, you may even realize that the goals that you’d set in the past are outdated and it’s the time to come up with new objectives. With the development of your company and alterations in your priorities, you may have to face new obstacles altogether. Hence, it would be recommended to re-evaluate the entire concept of objectives and goals.

Examine Qualitative & Quantitative Data

Now that the objective of the website is clear, then comes the time to examine the current situation. At this point, you must have your keen attention secured to find out answers for questions like: What is your current position? What is the place you’d want to be in the next couple of months or years? Which website elements are affecting your growth negatively? What are the website elements that you can optimize? And so on.

This is the point where you should be utilizing the available data and apply it to analyze the measured step taken to fulfill objectives. Once you have analyzed the required information, then comes the time to build the first hypothesis that you’ll be verifying subsequently.

Develop the Process & Test Scenarios

So far, you’d be well-aware of your objectives and the ways through which you can accomplish your goals. Now comes the part of technicality and methodology. For this, you must be pondering upon certain elements, like Which sides and areas you should be testing? What are the tools that you can install? What is the required traffic volume?

One of the most common mistakes that usually make in this step is utilizing such tools that aren’t competent enough to assess performance indicators set by you. So, make sure that you remain extra cautious in this aspect.

Preparing for Technical Aspects:

This is the stage where you will begin testing the variations of graphic design, adapting them to the SEO standards and installing the required tools for measurement. Once you have completed the technical preparation, it is essential that you verify everything that you’ve installed correctly before you launch the website.

Also, you must keep in mind that one simple error during the experiment may cause a huge change in the website. Since the outcome can be dreadful, hence, spend at least an hour verifying everything before you move ahead.


Indeed, it’s one of the most pleasant parts of the entire testing procedure. In this stage, you experiment with variants, monitor the performance, alter according to the results, and accomplish the desired outcome.

Analysis of Result

Once you are done with the testing, then comes the time to analyze results. Find out answers to questions like Have your actions did justice to your objectives? What modification brought significant result? Is the hypothesis complete?

In this way, you will also get to learn several new things about your users. The variant that will be having the highest rate of conversion will be your best bet.

A/B testing ideas for e-commerce websites with examples


“Look for the elements that could yield your biggest wins, and test them first.”
Kathryn Aragon

Undeniably, you cannot make everything work on your website. That’s why you conduct the test to know what is working with the help of data.

That being said, when it comes to an e-commerce website, it is important to be cautious of how and what trends leave a positive impact behind and provide an amazing experience to customers. So, if you own an e-commerce website, here are some of the best test ideas that you can implement on your website.

Pay Attention to Appealing Products

Out of all the objectives that a website’s homepage has, one of the most important ones is to compel people to get off the homepage and visit other ones. And to get it done, the best way would be by providing an amazing offer that might interest them.

For this, you can consider your best deals or best-selling products, create a deal out of them, and showcase them on your homepage at the top.

See how eBay has displayed their Featured Collection to attract their visitors toward them.

Pay Attention to Appealing Products

Search Bar Is Important:

If you are selling products that you would want people to search and find, you must be making the search bar bigger and give more prominence to it. Making it the center of attraction will bring amazing results for you.

See how Flipkart has managed to keep the search bar on the top. Even if you scroll down, the bar wouldn’t go anywhere.

Search Bar Is Important

Add Benefits of Purchasing from Your Website:

When talking about the e-commerce industry, undeniably, Amazon is the king. So, if you are starting up, you must add up what are the reasons that make you stand out from the rest of them. In this situation, you can add a bar including the benefits of purchasing from your website.

Look at how eBay has specified it. It is a clear, precise and alluring banner that simply tells why you should be shopping from here.

Add Benefits of Purchasing from Your Website

Always Keep Your Product Description Game Up:

If you think product descriptions aren’t worth it, you are wrong. They do matter. A lot. The product copies are meant to provide information to buyers so that they can easily persuade themselves that buying the product would be a perfect idea.

Look at how Flipkart has managed to put an in-detail description of its products.

Always Keep Your Product Description Game Up

Also, you have such products on your website that you haven’t manufactured, you shouldn’t, in such a case, repeat the pre-written descriptions. Add your personal touch and tell your customers about the advantages of the product.

Product Videos Will Be Effective:

Although there is no denying the fact that images attract more people, but videos keep them hooked. In a way, images are restricted; however, videos take your customers to the next level where they can peruse the product in a better way. If you haven’t thought of this idea yet, then try it right away. Begin for some products and examine the difference.

Amazon has videos for almost all of its products. They not only tell about the features but even provide usage information.

Product Videos Will Be Effective

A/B Testing Using HumCommerce

When it comes to a/b test, there are several tools available on the internet that you can go through and use. However, not every other tool will be providing you enough results. HumCommerce allows you to do a/b testing is a very user-friendly way.

A/B testing is also known as experiments.

Creating an experiment

To create an experiment, go to Experiments => View Experiments and click on “Create new experiment” in the right corner. The form will ask you to enter some basic information to get you started with your experiment quickly.

Viewing experimentsDuring the whole process of creating and running your experiments, HumCommerce will show you help text explaining what each element means. This is especially helpful when you have not run experiments before.

Adjusting an experiment to your needs

Once you have created an experiment, you can adjust it to your needs. We recommend to first have a look at your Success metrics.

Adjusting an experiment to your needs

Comparing the performance of different Pages URLs

In case you want to experiment and compare the performance of different pages URLs, you can create your experiment where you specify a Redirect Page URL for each variation. When users land on the original page URL, they will enter the experiment (and be assigned one of the Redirect URL variations). Then they will either: stay on the current page (if the Original variation was picked), or be automatically redirected to the Redirect URL they were assigned. When you create an experiment using “Redirect URLs”, please make sure to include the HumCommerce Tracking code on all your pages including on the Redirect URLs pages.

Starting your experiment

To start your experiment click on “Embed code” and embed the displayed code into your website, app, server or campaign. The experiment will automatically start running as soon as the first user enters the experiment. Alternatively, you can schedule a “Start date” by clicking on “Schedule”.

Finishing your experiment

Once you have run your experiment for a while and came to a conclusion about your results, it is time to finish your experiment. An experiment can be finished by anyone who has at least Administration access for a specific website. To mark an experiment as finished, either click on the link “Finish experiment” on the top of your experiment report page (in the experiment summary), or click “Finish experiment” when editing the experiment in the “Manage experiments” section.

Once you have finished your experiment you should remove all code that is related to this experiment from your website or app.

You will still be able to view the report for your finished experiment as long as you have not archived it.

Archiving your experiment

When you archive your experiment, the reporting page and segments for this experiment will no longer be available in the HumCommerce reporting UI. This helps to keep the reporting UI clean and to focus on your new experiments. Once an experiment has been archived, you can still view it in the “Manage experiments” section by choosing the status “Archived” in the filter above the list of all active experiments.

An experiment can be archived by anyone having at least Administration access for a specific website either directly on the experiment report page in the experiment summary or when editing the experiment in the “Manage experiments” section.

Apart from a/b testing, you can record visitor sessions, generate heatmaps, and do much more to analyze and optimize your ecommerce website.


Now that you know how convenient as well as effective this a/b testing is, you wouldn’t stay behind when it comes to conducting a test, isn’t it? When the question arises for how to do a/b testing, the entire process might seem easier as well. While it is important to conduct the test, it is even more necessary to do it in the right way.

So, use the points mentioned above while conducting an a/b test and get the most out of your input. Make sure that you are avoiding the common mistakes while testing and experimenting to gain the maximum result.