Essential Guide To A/B Testing

Essential Guide To A/B Testing

4 minutes read

Habil Emmanuail Sep 18 2019

What is AB Testing

AB testing, previously known as randomized controlled trials, is a way of testing that has been around for more than a 100 years. Back then it was used to test out different fertilizers on different patches of crops to see which had the best result.

Today it has evolved into AB testing and it can be applied to anything from a website,  to an application, to a video game.

Consider A (control) as the original web page or app and B (variant) as a variation to that. The variations can be as many as you like. The purpose is to test out all these variations to get a definitive result to what works best- which design/ layout gets the best response from the users.

Importance of AB Testing

In the past, companies would work on intuition to what customers wanted and how they reacted to the visuals they saw. The problem is that nobody can predict how viewers will react- the viewers don't even know it themselves unless they are faced with it.

That's where AB testing comes in. Instead of making guesses, you do a test based on your hypothesis and get a sure shot result of what the customer needs. You find out exactly what visuals bring out a better response from people and a higher conversion rate.

How it Works

Optimizely, Firebase and VWO are a few of the platforms offering services for this experiment, but if you are new to the market and don't want to spend money on AB testing, you can always use Google's Optimize (which is free). These platforms provide a dashboard of statistics which makes it easy to compare the results of your experiment in real time.

Let's take the example  of a web page. You create and upload your original website onto the internet. You make it clear for googlebot as to which  is your original website. It will overlook the variants in search results. However, it is still recommended that you don't run your test for longer than needed so that you don't create misunderstandings for the bot. This can affect your ranking on search engines.

It is considered wise to wire it up to remote config at the beginning so that you can change values without actually affecting the original version.You can make simple button changes or create a completely different layout in your variant design. Your headline can be in a different color or have different text to find out which attracts more viewers. The images can be different. The placement of objects can also be switched around to see what viewers find more convenient.

Working with one element at a time is one way to go about things. You get to know exactly which elements work and you can combine those winning elements to create a  final look.The optimizing app sets up both control and variant pages for you and starts giving data in a dashboard format. You can set how much traffic you want directed at both pages. It could be 50-50 or 70-30 or 60-40 or whichever ratio you prefer.

In your dashboard, you get to know how much time has passed in the experiment and get to know how many visitors you received in that time period. Then there are detailed figures for both the control web page A and variant web page B.

These figures include an overall conversion rate and individual figures for the elements that we changed.All the data is also presented in the form of a graph to visually show any changes in the figures over time in respect to the number of visitors to the website.

Now the person deciphering the data needs to keep all factors in mind and compare the results to know if that 2% change can actually  be considered a higher conversion rate. So how do we know how long to keep on testing and when it's the ideal time to stop testing and start deciphering the data? It depends on situation to situation and the kind of website or app you are testing. Some experts recommend to at least wait a week into the experiment so that you have enough feedback from viewers. Obviously you can't consider data from just 20 people. You need a much higher audience for better results. But also bear in mind the point we discussed earlier about keeping the test alive too long- it can confuse the bots as to which is the actual website and affect your search ranking.

Alternatively you can do an AA test before the AB test to rule out the possibility of any false positives later in the AB test.These same experiment techniques can be applied to applications and games to find the highest conversion  rate.

Is AB Testing for you?

If you wish to be successful in any business, you need to know your target audience and know exactly what they require. As with any business development plan, you do your homework- you do research and find out exactly what your audience wants.

So if you want your website, app or game to be successful, it's a wise option that do your research and in current times, AB testing is that research. In the fast changing technological world, the future might have a different way to do such experiments and do research on what your audience wants, but right now AB testing is the best thing to bring you success.

Habil Emmanuail

Habil Emmanuail

Habil is an IBM certified Design Thinking practitioner and a member of the Interaction Design Foundation. He has been working as a UX consultant and User Centered designer across web and mobile applications for almost four years. His area of expertise include UI/UX Design, UX Research, Visual UI Design, Prototyping, Information Architecture, Agile Scrum, Branding as well as HTML/CSS, UX Writing, Research and Analytics and Strategy and Planning.