Useful Stuff

Split Tests. Give Them 7.

But have you ever asked yourself how long you should test for? We have.
 
Why? Well because if you run a test for too short a time period you may not get enough visitors to validate the findings. You also run the risk that the findings may not have longevity, and won’t hold up over a longer period of time.
 
And if a test runs for too long there’s the danger that you’re diverting traffic away from pages that may perform more effectively, wasting time and money as a result.
 
Conversion rates can also vary greatly depending on the time of day, the day of the week, the season etc so we don’t want our results to be skewed by these fluctuations.
 
Most analysts will say that split tests should run for at least 7 days.
 
Imagine one of your variants provides a whopping conversion increase early doors, but then this plummets after 6 days. The only way to assure that this fluctuation is accounted for is to allow your test to run for a longer time period. It’s obvious really. Don’t get impatient and select a variant too quickly. It might not actually increase your conversions in the long run. Epic fail.
 
So the most important rule to remember is to give your tests long enough to evolve, to provide you with solid conclusions giving you confidence that the outcomes are accurate (and awesome).
 
Don’t peak too soon. It won’t make you popular.
 
©Copyright 2013 The ConversionArium Ltd
Call 0800 612 7164