Running Multiple A/B Tests on the Same Landing Page
If you want to optimize two separate elements on the same page, there are a few things to keep in mind.
On miacosa.com, I currently have two A/B test taking place. The most obvious one is the hero shot where I am either showing Wrigley field (Go Cubs!) or the SF Giants stadium.
The second test is at the bottom of the page where I have an A/B/C/D test optimizing Google text and image adsense. Recipe A is two text ads, Recipe B is text on the left/image on the right, Recipe C is image on the left/text on the right, and Recipe D is two image ads.
If I ran these two optimizations (or campaigns) independently and concurrently, the resulting data would be flawed. That's because if someone saw Wrigley Field (Recipe A of the first campaign), they could have seen any of the four recipes of the other campaign. A person's actions are not influenced by single elements acting alone - in other words, which recipe a visitor sees in the campaign at the bottom of the page influences their actions just as the recipe at the top of the page does. To be valid, the campaign needs to ensure that it takes into account how each element being tested affects the other elements.
Here are three quick ways you can approach a situation like this:
1. Combine both campaigns into one. The new campaign would have 8 recipes or would be an A/B/C/D/E/F/G/H campaign. This way the results reflect all the possible experiences a visitor may have.
2. If you have a decent level of traffic, use profiles to segment out the traffic. Profiles are powerful and allow you to do amazing things like behavioral testing. Essentially, profiles represent a way you can tag your visitors and serve differing content to each group.
In this example, I would tag visitors immediately upon their arrival as either ‘group A’ or ‘group B’. I would then target one campaign to ‘group A’ and the other to ‘group B’. Group A will only see one of the variations for the top campaign - the bottom campaign will be the same for every Group A visitor. Likewise, Group B will only see variations of the bottom campaign, while the top campaign will be the same for every Group B visitor. This way if a visitor gets into one campaign, he doesn't get into another, and you get clean results.
3. Turn the campaign into a multivariate test. Multivariate tests are a very powerful tool when it comes to optimization. By turning this into a multivariate test, you will get element and alternative analysis, which gives you insight into what elements and, more important, what alternative is having the greatest impact towards conversion.
The above ideas are from Brian Hawkins' Miacosa.com blog.
