by Matt Pezzimenti
April 1, 2020 • 5 min read
A/B Testing is one of the most powerful and influential components of a Conversion Rate Optimisation program.
It enables you to remove stakeholder “best-guesses” about the right road to higher conversions and replace them with proven strategies based on audience actions.
What is A/B Testing
But what exactly is an A/B Test, how do you conduct accurate and insightful tests, what happens once a test closes and how do you apply winning results to a live site?
An A/B test presents part of your audience with an original (control) version of a web page and a modified version (variant) to an equal number of visitors. The objective is to learn which variation better achieves a specific conversion goal e.g. more leads, sales, memberships, subscribers, etc.
Precise and methodical A/B testing is the lynch-pin to executing controlled CRO programs.
It is important that you:
- – Create goals based on qualitative user research and quantitative data
- – Only change one element per test so it’s easy to isolate changes that make the biggest impact
- – Combine multiple winning test results and see ongoing measurable improvements
- – Discover why certain elements impact people’s behaviour and use it in the future
- – Gain invaluable information even when variants lose.
How to run A/B tests
In a CRO program, A/B testing quality is key to the overall CRO testing findings. Every step of your CRO test execution needs to be carefully controlled. Otherwise, you risk losing or missing valuable data.
At Conversion Kings, a large part of our industry-leading conversion results is due to our detailed testing process that delivers more accurate, revealing and beneficial results.
Here’s a snapshot of what’s involved in executing an A/B test:
Specify the goal
Using the research conducted in the CRO audit stage, and learnings from thousands of previous tests, we develop a hypothesis that clearly states the change being made, predicted results and reasoning behind it.
UX Prototyping and CRO test development
Taking hypothesis goals, website restrictions and audience needs into consideration, we create test page mock-ups for approval before coding a precisely replicated test design.
User acceptance testing
Our in-house UAT experts check for any technical problems that could sway or unravel the A/B Test. If you conduct tests without UAT it could cost you time, money, resources, and business.
Developed A/B tests are then reviewed again by an Account Manager and presented to the client for any feedback and revisions before final approval is given.
Launch the test
When the test goes live, CRO tracking technology examines the effect of the test variant against the live control web page.
Monitor and discuss
Throughout the test, we closely assess how variant compares to the control condition on key conversion goal metrics, and bring test experts together with the client to discuss the latest findings in weekly meetings.
Close the A/B Test
The test runs until 95% statistical significance has been achieved. This figure refers to the likelihood that the results we get are due to the test variations and not because of randomness or other reason.
What happens when an A/B Test ends?
Unlike most A/B Testing, where a single test or random tests are used to find a winner, Conversion Kings CRO testing is strategically ordered into rounds.
When each A/B Test closes, we meet to answer, “What did we discover in this A/B test?”, closely analysing data from Google Analytics, heat maps, and click maps.
The results and learnings are used to develop a new hypothesis and enhance the next round until we uncover an ultimate experience.
By using this trademarked Series Testing approach we can:
- Know for certain which test works best
- Reduce production costs and avoid wasting time on tests that don’t advance progress
- Keep all stakeholders updated on findings
- Closely predict how much additional revenue the ultimate winner will generate
Comprehensive A/B testing as part of a CRO program provides many core-business, digital marketing and design benefits.
In addition to uncovering proof to make decisions that help maximise ROI from existing web traffic, A/B test changes reduce website bounce rates (good for UX and SEO), enhance customer feelings about your brand, and ensure website redesigns are risk-free and profitable.
The Conversion Kings team have conducted over 8000 CRO tests to help client’s achieve conversions at 6 times the industry average.