Let's talk about A/B testing for e-commerce businesses. So A/B testing is the world's most widely adopted data-driven approach. It's about splitting your traffic, splitting your visitors and showing different messages to each segment of your visitor.
Usually we call the original version control and the other tested version variation or challenger version. There are multiple types of A/B testing.
The classic A/B testing is just splitting your visitors into two groups and serving 50-50% different messages.
We also have A/A testing which is basically splitting the visitors into two and serving them the exactly same message and exactly the same content. Why do we do this? We usually just do it to test the platform and the software and the architecture if everything is working correctly.
We also have what we call A/B/n testing. It's about serving like A/B/C, A/B/C/D, splitting your traffic into more than two different parts. But the technique and the mathematics behind it is exactly the same as with A/B testing.
And we also have what we call multivariate testing which is about testing multiple things on your website at the same time and testing all kinds of combinations of these different elements on your website.
We can also categorize A/B testing depending what you are actually testing.
When we usually say A/B testing, we often understand the page or web page A/B testing. Having a landing page for example and splitting the traffic that goes to that landing page into two different parts and serving them two different versions of that landing page.
Another part is what we call popup A/B testing, basically having different versions of the overlays of the popups and seeing which one converts better.
And we also have what we call multi-campaign A/B test or journey A/B test when we actually split the visitors into two parts and for one part we show a totally different journey, different pop-ups, different landing pages, different messages than to the other part.
This is the most complex A/B testing technique that you can run on a website.
So most tools that serve small and medium businesses usually use the Bayesian algorithms. And there's another term which you will probably meet if you dig deeper into conversion optimization which is called Statistical Significance.
So A/B testing is very much about chances. When we A/B test a website or a popup for example, we will see that one version performs better than the other. But this better performance, this uplift can be due to the changes that we actually made or it can be due to chance and just pure luck and randomness.
So basically we can never have a 100% probability that one version is better than the other. We can usually just say that there's a 99% chance, for example, that the challenger is better than the control.
