Split-Testing : The Secret Weapon of Experienced Online Marketers

Since the early 1900s, the best direct marketers have used some form of split-testing to improve their sales messages. Marketers like John Caples split-tested with coupon codes to tell which of a number of separate ads produced the best response, changing one item at a time until they found a winning formula. Split-testing is how direct marketers continually increase their profits over time.

With the ease of tracking on the Internet and the advent of low-cost or free software programs, split-testing is something that all online business owners should be implementing each month. It is an ongoing process.

What Is Split-Testing?

Split-testing is the process of creating two separate variations of anything that a visitor or a prospect is exposed to which requires them to take an action, such as a web page, a PPC ad or a banner ad. An action required for measuring a split-test can include the click-through-rate on a PPC ad, submitting a form to request a free consultation or purchasing a product online. Splitting traffic between the two variations of the same element shows you which one produces the best response rate.

Split-testing allows you to remove the guesswork from your marketing campaign by showing you the difference in response rates each variation produces. The first time that you design your web page and set up a PPC campaign, you’ll of course have to use your “best guess” and experience helps a lot here. However, once a page or an ad is activated and you start achieving results you can use split-testing to continually and incrementally improve these results over time.

Essentially, split-testing allows you to identify the combination of factors that produces the highest conversion rate and, in turn, the highest return on investment (ROI) from your advertising budget.

Why Is Split-Testing Important?

Split-testing is the fastest way to increase your conversion rate and lower your cost-per-action, which increases profitability. But more than this, it allows you to be more competitive in your marketplace by allowing you to spend more on advertising than your competition, yet still achieve higher levels of sales and growth at a lower Cost-Per-Action (CPA).

Split-testing also allows you to test what your competition is implementing on their site and to see how it affects your own conversion rates. Over time, split-testing allows you to capture a larger share of your online market.

Let’s Take A Look At The Numbers

Your competition is spending $2 per click and has a 1% conversion rate. That means it costs $200 to drive 100 website visitors. At a 1% conversion rate they are getting 2 conversions (ie sales or sales leads) for every $200, which means their CPA is $100 per action.

If through split-testing you were able to increase your conversion rate to 3% that means for the same $200 spend you would get the same website traffic, but instead of getting just 2 conversions, you would get 6. This means that your CPA is $33, rather than the $100 your competition is achieving…all from the same advertising budget.

What this really means is that you can afford to spend 3 x what your competition is spending and achieve the same CPA, which means that you will dominate the top positions on the search engines and capture the lion’s share of the online market. Convinced yet?

Conversely, if your competition is split-testing and you are not, they will get the lion’s share of the online market and you will miss out.

In the current economic climate, split-testing allows you to get the most out of your advertising budget and to continue growing your business during these tougher times.

What Should You Split-Test?

You should split-test anything shown to a visitor that requires an action. With PPC advertising you can split-test two separate ads and see which ad produces the lower CPA. With your website design you can split-test two separate pages and see which one produces the highest number of conversions. You can also split-test email campaigns, banner campaigns and other forms of paid advertising.

SEO listings (ie what displays in Google’s organic search) are harder to split-test because you don’t have full control over the information displayed in the listings. Saying that, you can split-test the page that prospects land on after they click on your SEO listing.

On your website, you should split-test the following items on your highest traffic web pages – which you can find through Google Analytics:

  • Headlines
  • Call to action
  • Offer
  • Sales copy
  • Images
  • Testimonials
  • Button styles
  • Navigation styles
  • Background color
  • Listing a phone number versus not listing a phone number
  • And anything else which holds a prominent position on the page

For your PPC campaign, you should split-test the text ads, keywords and landing pages.

How To Split-Test

The most important metric to measure with split-testing is the CPA, which is measured with conversion tracking. If you don’t have a website form or shopping cart and all your website leads come via telephone or email, it will be much harder to split-test.

The easiest way to split-test is to use a form on your website for somebody to submit whenever they wish to contact your company. When somebody submits that form, they are taken to another page that is call the “thank you” page, which contains a short message saying something like, “Thank you for your enquiry, we will contact you shortly.”

With split-testing, a conversion code is placed on the ‘thank you’ page which then measures the results. If we were split-testing two landing pages using different headlines we would use split-testing software to rotate between those 2 pages so alternate visitors would see alternate versions of the landing page. After a reasonable amount of visits and because conversions are being tracked, one of the two versions will start producing a higher number of conversions than the other. This is then the better landing page, also referred to as the ‘control piece’. The goal with split-testing is always to try and beat the control piece.

To justify that a split-test is statistically correct you need enough visitors and conversions to make an informed decision– as a general rule you will need a minimum of 30 conversions before you can make a decision on which of the variations is best. Note that The difference between the conversions of each version has to be greater than the square root of the sum to ensure statistical integrity.

Split-Testing Software

The best software to use, at the moment, for split-testing web pages is Google’s Website Optimizer, which is available for use free of charge. With this software, you can split-test one or more pages on your website at the same time. It is a very robust system, and the software actually tells you when you have found a ‘winning combination’.

It’s relatively simple to setup and although it can be a little confusing the first time, it quickly becomes very easy to use. Setting up a new split-test can take as little as 15 minutes (after the two variations of the page have been designed).

Split-testing of PPC ads can be done directly through the Adwords interface by setting up a variation of an ad within the campaign. Just make sure to edit the campaign settings and select the ‘rotate ads evenly’ rather than ‘optimise ads based on performance’. Importantly, you will need to activate conversion tracking within the console and add the conversion pixel to the ‘thank you’ page.

Split-testing other forms of marketing is done in a similar way to split-testing PPC ads, that is, managed from within the administration console of the advertising channel and tracked by adding a conversion pixel to the ‘thank you’ page.

Additional Tips for Split-Testing

  • You should always be split-testing : With split-testing you set up the test once and let it run for a few days, weeks or a month, depending on your level of traffic. We recommend that as a bare minimum you split-test all of your high traffic web pages and all your PPC campaigns at least once a month. This means that once a month you sit down and review the stats, stop the lower performing variations, and then setup another split-test to try to beat the control piece.
  • Track your split-tests : It’s important that you track the changes that are made so you don’t end up split-testing the same thing over and over again, ensuring that you’re continually moving forward. Track changes with a simple spreadsheet, with columns for what is being split-tested, the date and the results.
  • Watch what your competition is doing : Learn from them but don’t take what they are doing and implement it on your websites as is… split-test it and see if it improves conversions or not.
  • Only make one change at a time : If something is working, only make one change and split-test that – for example the headline. The exception to this is where something’s not working, then you can split-test two very different sets of ads or web pages and see which one produces results.

Remember that split-testing is the key to ongoing success; it is very low cost compared to the benefits you can achieve. In some of our recent split-tests we were able to increase the conversion rate of a website by more than 30% by improving the design and layout of the home page. And this took us less than one-day to implement. Imagine getting 30% more sales from your existing advertising budget…

Alex Cleanthous

Alex Cleanthous

Chief Innovation Officer at Web Profits

Alex Cleanthous is Chief Innovation Officer at Web Profits. With more than 10 years experience in online marketing, Alex is always on the lookout for smarter, faster and more scalable ways to achieve maximum growth with minimum spend.


  1. David Duffield says:

    Another very good article Alex and hopefully your next one will cover Google Analytics so that when we do the split-testing we can learn more about how to monitor the conversions for each test.


    April 22nd, 2009 at 4:36 pm

  2. Alex Cleanthous says:

    Hi David,

    Conversions for each test are tracked within the Google Website Optimizer administrative console.


    April 23rd, 2009 at 10:27 am

  3. Giovanni says:

    I am quite new to this and just today I discovered the existance of this Google tool. Your post will provide good help. I cannot wait to run my fisrt test. I was thinking about starting with a headline test. In each marketing webinar / ecourse they quite only talk about that.
    Anyway, for me it’s just a beginning.
    Thanks again.

    April 23rd, 2009 at 9:48 pm

  4. Shaun McGowan says:

    Some good info in here, easier to split testing now with PPC and different landing pages etc. Still cant beat organic results, if we can learn to master Google’s algorithm and stay on top of it, we will be winning!!!

    August 18th, 2009 at 8:38 pm

  5. Can Split Testing Improve Your Real Estate Ad Response Rates? says:

    […] is Split Testing? Split testing is an extremely powerful strategy for increasing the effectiveness of your marketing. You can use […]

    August 30th, 2009 at 6:50 am

  6. Gino says:

    Indeed, very important to increase your website’s effectiveness.

    I have done this using Logaholic web analytics (www.logaholic.com in combination with a lot of other in-built features they provide.

    Very helpful

    September 16th, 2009 at 10:54 pm

  7. How To Use Google Analytics To Increase Sales : : Web Profits says:

    […] that your visitors browse to after visiting your home page. These are the pages which you should be split-testing using the Google Website […]

    October 13th, 2009 at 10:56 am

  8. link building india says:

    Good Post, I think split test is powerful testing to drive traffic.

    December 9th, 2009 at 3:21 am

  9. Rex Dixon says:

    Would you have time to share/upload some results so others can see some examples of your work over on A/B Tests dot com?

    April 22nd, 2010 at 1:40 am

  10. What Is A Good Conversion Rate? says:

    […] Before you can assess your conversion rate you’ll first need qualified prospects visiting your site. When that is happening, the best way to assess how good your conversion rate is to ask yourself ‘am I generating a profit from my website visitors?’. If you are then your conversion rate is good, if you aren’t then your conversion rate isn’t good. And if you aren’t then you need to work at improving your conversion rate through either a complete overhaul of your website design or through ongoing split-testing. […]

    August 19th, 2010 at 9:02 am