A/B split test with Sitecore Online Marketing Suite | Lars Birkholm Petersen

A/B split test with Sitecore Online Marketing Suite

In my previous posts I have mainly focused on Behavioral Targeting, as I see this as one of the strongest actions to get more value of a website.

Another and very used way to optimize for better conversions is A/B split testing, which let you test different content on your site and optimize for which gets most conversions!

You may think, that you know which content is the best; on the front page, as teaser spots or as call-to-actions, but why not let your visitors decide?

.. And yes you can have them both, just imagine what Behavioral Targeting with a twist of A/B split testing can do for your conversions.

With Sitecore Online Marketing Suite, you can configure and run several tests on your site. But a good rule is only to run one test at a time, so the different tests won’t influence each other.

It’s really easy to get started – and in a short while you won’t be guessing about what works best on your site, but you can recommend actions upon real test data.

So first up you need to have some content to test, fortunately there’s a lot, like:
- Front page
- Spots on front page
- Spots on pages
- Call-to-actions like newsletter sign-up
- Forms like contact us (is 5 fields better than 6 and how many more conversions do we get)
- Design changes
- Etc-

Lets say I have a Camera brand site and I want a spot, where the user can click and see offers for pro digital Cameras.

I can create different spot and test those, in my example I have created two spots – one red and one dark, see below:

Red spot test variable in Sitecore Online Marketing Suite

Dark spot test variable in Sitecore Online Marketing Suite

In order to setup the A/B split test in Sitecore Online Marketing Suite, I open the Marketing Center, right click on the “Test lab” section and insert a “Multivariate test”, see dump below.

Create A/B split test in Marketing Center

I give it a name (in this example “Pro offer spots”) and just need to fill out a description and chose whether the test strategy should be “Random” or “Sticky”, meaning that random can show different test variables to same visitor, while sticky only shows the same test variable.

Configure A/B Split test

Then I add my test variables, by right click on the test I have just created (Pro offer spots) and insert a “Multivariate Test Variable”, see dump below.

Insert test variable in A/B split test

For each test variable I give it a name and select the data source, where my content is in the Sitecore tree. In the dump below I have created a “Red BG with best offer”.

Select A/B split test variable from Sitecore

I create all the variables I wish to test, in my example I have just created two – one with red background and one with dark background.

Once the test is created, all we need to do, in order to use it, is to insert it into the page(s) where we would like to have the spot.
This can be done by using the Page Designer, where I can add the rending for tests, in my example it’s called “Flexible Personalization Spot”. See dump below where I have added the rendering to the right column in a three column layout.

Add A/B split test rendering i Page Designer

With the rendering selected I open the Properties and select the “Pro offer spots” from the Test lab.

Setting the rendering properties in Page Designer

And voila – you have created a A/B split test on your website.

Whats left is to wait untill there is statistical significance, look at the build-in test report and see which test variable performs best.

With the test report in Sitecore Online Marketing Suite, tests is measured against your site goals – so instead of looking at how many clicks each test variable gives, you can see the outcome!

For more on renderings and page designer, please read How to get started with Behavioral Targeting (Part 2).

If you want to be inspired and see what other have tested, try to visit www.abtests.com, which has a lot of examples with outcomes.

Feel free to comment, if you have anything in mind!

Bookmark and Share

Category: A/B split test, Online Marketing, Sitecore | Tags: , , 2 comments »

2 Responses to “A/B split test with Sitecore Online Marketing Suite”

  1. Erick Mott

    Lars, I just read your recent white paper “Why Website Personalization Matters” and like your thinking about website engagement management; blogged about it here:


    One thing that continues to amaze me, and according to some research I read earlier this summer, the majority of online marketers don’t do simple segmentation and personalization work like A/B. Not a good idea from an end user perspective and when considering how important it is to maximize budgets by being more efficient and relevant.

    I look forward to more of your white papers and blog posts.


  2. Harpreet


    We’ve been trying to figure out how to use the Sitecore’s Online Marketing Suite for running an A/B test and your blog has been VERY helpful.



Leave a Reply

Back to top