When it comes to business, what do you think matters most? Having an amazing team? Lots of inventory? World-class technology? Well, all things aside, what matters most for a business is its people.

And how do you know how many people are interested in your business? You measure conversions, of course. After all, the number of clients you have is what makes you a successful business.

If you heaven’t already check out The Complete Guide to A/B Testing (Part 1) before reading this post to learn the conversation testing fundamentals. 

Without conversion, even the best of businesses will fail, every product would be worthless, every patent useless. Consumers are the reigning kings and queen of a business and everything is at their disposal.

What differentiates the successful businesses from the less successful ones is their understanding of their customers. Among other ways, A/B testing is a great way to measure what your customer wants.

A/B testing is the direct way of measuring how a customer responds to the different variations of a web page and the results could be used to push the ones that resonate with the users most, which in turn can help in increasing conversion.

Check out the post below to understand why you need to incorporate A/B testing in your business, the things which can be tested, the guidelines for you to start practicing A/B testing and some sample case studies.

This is the second and last post in the two-part series about a complete guide on A/B testing. If you haven’t yet gone through the part 1 of the blog post, you may want to check that out first here.

Introduction to A/B Testing

Do you feel like Penny in the world of Sheldons? When every other marketer talk about analytics and A/B testing, do you feel like you are sitting in a Chemistry class? Well, A/B testing has nothing to do with Chemistry and it isn’t a complicated subject at all.

A/B testing is just what the name represents – it is a process to test binary version of A and B, to find which one works better in terms of conversion.

Still not sure how it works? Imagine yourself as a baker in pursuit of a perfect batch of blueberry muffins. How do you ensure that you have the best muffins? By baking few batches and then comparing to see which one comes out nice!

A/B testing does exactly the same! It takes an item, test few variations against the original and then compare and evaluate and see which version performs best and provide better conversion rate.

A/B testing can cover websites, landing page, emails, advertisements and even applications.

Why should you A/B test

A/B testing takes the guesswork out of a campaign. It allows you to make changes to the user experience and collect data for the results. It helps you know exactly how a change may affect your business performance. You can create hypothesis and learn why and how certain elements help in impacting the user’s behavior.

It can be used to improve the user experience consistently, ultimately leading to the optimization of conversion rate.

For instance, if you are creating a landing page to get more subscribers for your blog, you can test the changes by editing headline, form fields, call to action, images and overall layout. A/B testing also helps you optimize your item for the desired outcome and can make your marketing efforts more effective.

With A/B testing, marketers can learn what kind of headline attracts the customers, what kind of ad-copy garners more clicks, what kind of images they like, what kind of layout and designs covert best. In fact, in the long run, the overall spent on the marketing campaigns can be decreased by optimizing each element to work efficiently to improve conversion rate.

As long as there is a clear hypothesis and the goals are clearly defined, you can always optimize any facet of your marketing campaigns to ensure better engagement and better conversion. 

A/B Testing Process

The correct way to run an A/B testing is to follow a scientific method. Here is the framework for A/B testing which can be used to start running tests:

Identify Goals

The first step for you should be to identify what are your goals for A/B testing. Your A/B testing goals should be based on your business objectives. Make the objectives as specific as possible.

These are the metrics which should be used to decide whether or not the variation works better than the original version and without any specific goal, you would not be able to pinpoint exactly how far better it works. The goals can be anything from more clicks on the call to action button, more sign ups, or more click through rate.

Ask yourself these questions and construct hypotheses. Is your website having higher bounce rate than expected? Are your sign ups really low? Then, calculate the number of visitors and days you would need to run your test for, that is, how long do you want to test the variation effects?

Collect Data

The best place to start testing to optimize is to look for campaigns or pages with low conversion or high bounce rates which could be improved. Begin with high traffic areas as it will allow for a faster data gathering.

Generate Hypothesis

After the goals are identified and you have pin-pointed high traffic areas, you can then start generating hypotheses to create variations.

How do you think a campaign could be improved on? Do you think a headline change is warranted? Create a list of ideas and then prioritize them according to their expected impact and the implementation efforts.

For instance, if you are working to create a variation for a landing page, do you think a headline edit would bring in desirable change in the conversion rate? Do you need to make major changes? Do you feel the colors and the layout needs to change? List out all the changes and then create variations for them.

Create Variations

Now that you already have a list of hypotheses, you can start creating variations. Always remember to make a variation with just one change at a time. Just because you think that headline and form field change can improve conversion doesn’t mean you create a variation with both these changes.

First create a variation with headline change and note the change in conversion rate and then create another variation with form field changes and then notice the difference. Always test against the control, instead of variations to variations.

There are a lot of software available which will help you make these edits easily like Optimizely, VWO, Unbounce, Google Analytics Experiments, A/Bingo, Adobe Target, KISSmetrics.

Run Experiment

Start experimenting to test your hypotheses. Through the software, visitors to your website will be randomly directed to either the control or the variation and their interaction will be measured and compared to determine how each experiment performs.

5Analyze Data

After the experiment is completed, you can analyze the results to understand how the control and variation works against each other. Are there any significant differences? For instance, if the change in headline doesn’t have any significant result on the conversion rate, then you can go back and construct another hypothesis to repeat the process.

Report results

Once you have analyzed all the data, report the results and the insight generated to everyone concerned.

What can you A/B Test?

You have created a nicely designed Landing page and you are ready to A/B test it! But you feel stuck because you are not sure, what to test? Should you change the color? Should you decrease the number of form field?

You keep scratching your head in dismay unable to come to a conclusion. Well, fret not, we have the perfect solution. Here is the list of things which can and should be tested when you are A/B testing. Now go on, test like a boss!


Headline is often the first thing that your users see on your Landing page and probably on some other collateral as well. You could A/B test between longer headlines vs. smaller headlines, between different styles of headlines, with different fonts and font sizes, with bold, italicized or underlined, with gradient vs. without gradients, with different positioning like (central, left, right, top, bottom, etc.), formal vs. informal kind of headline.

Have a look at the case study to understand how a small change in the headline yielded an increase in sales by 89.97%.


Specifically developed to counter joint pain problems, Movexa is a leading Joint Relief supplement from Vitamin Boat Corp which manufactures a range of natural supplements across US and Canada.

Test Hypothesis

According to the hypothesis, the word ‘supplement’ was added to the headline to bring more clarity to the landing page and improve the conversion rate.


The result was a whopping 89.97% increase in the conversion rate.

Key Takeaway

The key takeaway from this case study is that even a small difference can make a huge impact in conversion rate. Keep your headline crisp and ambiguity free. Let it be detailed so with just one glance itself, people can know what you are talking about.

In this case, the Control headline was “Natural Joint Relief’ which could mean anything from naturopathy, to exercise program to anything related. However, adding the word ‘supplement’ made the headline clear and removed the ambiguity. Remember even if people don’t have time to go through the details of your campaign, almost everyone reads headlines and making it descriptive, easy to read and interesting can offer value to your customers.





Subject line

Subject line is very important especially when it comes to email campaign testing. More often than not, subject lines are the deciding factors in getting clicks.

You can experiment with so many different kinds of subject lines to help you understand which subject line brings the highest CTR for your campaign. Short vs. long, informal vs. formal, funny vs. serious, with offer vs. without offer, urgency, ambiguity, etc., create various variables to see what kind of subject line resonates with your users.

Have a look at the case study where a change in the subject line resulted in an increase in the CTR by 2.57%.


The retailer wanted to leverage the holiday season to send email blast to encourage their customers to browse their collection.

Test Hypothesis

The test hypothesis was to understand which subject line gets the most click throughs between ‘Happy Holidays’ and ‘Merry Christmas’ or a combination of the two.


The subject ‘Merry Christmas’ had a 2.57% higher CTR than other subject lines.

Key Takeaway:

Don’t assume. Test different things as the result could be eye-opening as is the case with the above case study.



 Call-to-Action (CTA)

Call to action buttons are probably the most important elements of your page. You can experiment with a lot of different ways to present your call to action buttons like editing the wordings like ‘Buy now’ vs. ‘Purchase’ or ‘Get it’, adjusting the shape, size and the color of the button, changing the button position to top, bottom, left, right, middle, etc.

Adding Action words like ‘Buy now’, ‘Free trial’, ‘Watch video’, ‘Add to cart’, etc. deciding on whether having one or multiple call to actions on the page.

Check out this case study to see how a call to action button increased the conversions rate by 62%.


Consolidated Label company which is into manufacturing custom printed labels for retail products, wanted to test a call to action button on their test page as their original page didn’t have any call to action button.

Test Hypothesis

The idea was that adding call to action button would increase the conversion rate.


The result was a shopping 62% increase in conversion rate.

Key Takeaway

Although an increase in the conversion rate was expected, this huge change was a pleasant surprise. This shows the difference a single call to action can make. Always add call to action buttons to your campaigns.

Here are the screenshots:






Offering promotion is not good enough anymore, how you offer it matters! Test different ways to see what makes your promotions more enticing. Try with different wordings, percentage off vs., flat rate, offering pricing like $49 vs. $50 vs. $49.99. Check to see if free shipping works vs. free shipping above a cap limit.

Check out the case study to see how adding a free shipping incentive helped increase order by 90% for a skin care company.


An anti-aging skin company called as NuFACE wanted to boost their online presence and ultimately sales.


Since, there was enough traffic and they knew that the customers were holding back from purchasing, they wanted to offer an incentive to see if it would increase conversion.


Adding incentive of free shipping helped increased the orders by 90%

Key Takeaway

Not offering free shipping is one of the major causes of shipping cart abandonment. Always offer promotions and incentives to see if it can boost your conversion rate.





Layout and styling testing ideas

Test the layout too! Check if visitors prefer single column body vs. double or multi column. Decide on if you need to go with just one page or multi pages. Check the color family too from different colors to different color scheme, light pattern vs. dark. See if the navigation bar placement has any effect on the user engagement by placing them left, right, top and down. Test with different font, font sizes, text block and typography too.

This jewellery website increased its conversion rate from 0.01 to 5% just by improving their website layout.

Check out the case study to see how layout changes in the product page helped a company to increase its revenue.


SmartWool intended to improve their conversion rate for their website.


The hypothesis was to see whether changing the design on the product pages would lead to any increase in revenue per visitor.


The result was an increase of 17.1% in average revenue per visitor.

Key Takeaway:

Keep your product page easy to navigate. Stick with best practices to ensure a seamless browsing for your customers.








Content testing ideas

Check if you should add more content or little content and whether bullet points work or whether paragraph styling works. Know whether you should go for SEO optimized pages vs. human readable texts. Test with no text vs. having text, whether the text should be on left right, top or bottom.


Michael Aagaard wanted to get more downloads on his ebook.


The assumption was that the time was a major deterrent and a barrier for visitors to download the ebook. He wanted to add a liner to let the visitors know how long will it take to read and apply.


Changing a single copy resulted in an increased conversion rate by 18.59%.

Key Takeaway

Understand what could be a deterrent for your prospective customers and take necessary steps to provide a better experience to them.


Form testing and social proofs

Decide on whether the form should be short or long vs. how short and how long. Also, check if adding social proofs like case studies vs. testimonials work!

Check whether adding different images work and whether the images should be small or big or taken from different angles. Whether the testimonial added should be text or video?

Run different tests to find out which one triggers the best performance metrics.

Check out the case study to see how a change in design of social proof generated over 69% more leads.


comScore, an Internet Analytics company, wanted to generate more leads from their website.


Although they already had social proofs on their website, they wanted to see if a design change could help them generate more leads.


Social proof with logo generated 69% more leads than original.

Key Takeaway

Adding social proof is not enough, how you add it matters. Add logo and other relevant details to make it look more authentic.





Getting started with A/B TESTING

A/B testing can not only offer you immediate relief but it even provides a trend that many other marketing methods cannot. That is, with A/B testing, you can measure the behavioral pattern which offers an insight to come up with solutions even in the future. For instance, if you have tested many times with more text vs. less text, and every time your users have preferred less text, when you are creating a page for next time, you can add only less content to it.

Guidelines for Effective A/B Testing

  • The first thing you need to do before you can test is understand the web page you have developed and find areas of improvements. Just randomly testing wouldn’t yield any effective results, you need to have a proper web page first. For instance, if you are testing a Landing page, follow the best practices when creating the landing page. Only after the Landing page has been designed properly, then you should start the A/B testing process.
  • Wear the thinking cap! Think on what changes could bring in the desired results! Does the call to action looks small? Is the copy boring? Build a list of hypothesis for you to test against the original version.
  • For A/B testing, it is the best practice to test one variable at a time as testing too many things will not give clear results. For instance, if you are A/B testing a headline make the variation only with the headline change and test it. Adding different colors, or changing the form fields, at the same time will muddle up the results as you wouldn’t know what caused the spike in the conversion, whether it was the headline change, color or the edit in the form field.
  • Suppose you want to test a landing page and then you decide to test the email campaign at the same time. How would you know whether the change was due to landing page optimization or because of email campaign? For an A/B test to offer effective results, it is best to test just one element at one time.
  • Although you can certainly A/B test on a granular level, you could also create two completely different version of a page and test them against each other to understand which one works better. This should actually be done before you start testing on different elements. This type of testing offers better results as you can test both on the macro and micro level.
  • Sometimes small changes can yield better results. Instead of making huge changes, try changing the color of the CTA, making the font size bigger, smaller, changing the copy slightly, or switching the placement of the form. These changes are much easier to test and measure.
  • When you are creating A/B tests, be mindful of how it affects the bottom-line of a business. Just knowing the percentage of conversion it offers is not enough, you need to know how it affects the other metrics like traffic, leads, CTR, sales number, etc.
  • Always test against control and variations, instead of variations vs. variations. Keep the unaltered version and then create the treatment or variation for it and then test.
  • You don’t need to limit yourself on testing just the colors or form fields. The sky is the limit when it comes to A/B testing. You can test with design, images, colors, form field, layout, content and even the font size, the size of the images, the different kinds of layout. You can test a lot of things but don’t lose sight on the end result.
  • You need to have set of audience which are equal and random to achieve cohesive and conclusive results.
  • Timing is vital. If you run the control and variation test during different days and time, you wouldn’t know if the results were due to timing or because of different templates. It is absolutely important to test the results at the same time as otherwise you may be left second-guessing.
  • Decide on what you want to achieve before you start your testing process. The significance of the results would help you decide whether or not you want to go with the variation or stay with the control.


A/B testing might sound like a chemistry experiment, it is not at all complicated once you know how to go about it and have the right tools at your disposal. If you are still having questions concerning A/B testing, get in touch with us and we will help you become one of us – A/B testing experts!

It’s not just your mother who wants to hear from you, we do too ;)! Hearing from you will make us so much happier. So, get in touch with us, may be!

We are social too! Visit us also on our social media platforms, will you! And don’t forget to subscribe to our newsletter for a weekly dose of awesomeness!

0 replies

Leave a Reply

Want to join the discussion?
Feel free to contribute!

Leave a Reply

Your email address will not be published. Required fields are marked *