The ABCs of A/B Testing
Here are some statistics about email open rates:
- Including an emoji in the subject line improves open rates by 56%.
- Words such as “free” and “percent off” tend to decrease open rates when they appear in the subject line.
- If the recipient’s name shows up in the subject line, open rates increase an average of 18.30%. (Source: Backlinko)
You might wonder how these bits of knowledge are arrived at, and the answer is A/B testing. In the case of the statistic involving emojis, someone looked at the open rates of emails with emojis in the subject line versus open rates where emojis were not present.
A/B testing is a critical part of building a successful marketing strategy. You don’t need to be an organization like Backlinko (which has access to data from many marketers) to conduct this type of testing. Find out more about A/B testing below, why it’s important for your business, and how to go about it.
What Is A/B Testing?
A/B testing is also called split testing. Both names give you an idea of what it is. You run a test where you split items into two categories (A and B) to see which one performs better. Here are a few examples of A/B testing to give you a better idea of what it is:
- A deli wants to know if customers prefer turkey from a particular vendor over another. They create sandwiches with turkey from vendor A and a separate set of sandwiches with turkey from vendor B. Customers provide feedback, rating the sandwich on quality, taste, and overall satisfaction. Sandwiches made with turkey from vendor B get much higher ratings, answering the question for the deli.
- Marketers are creating a landing page for a new subscription service. One marketer thinks a shorter landing page will be more powerful than a longer one, but the team disagrees. They create a short and long version and drive traffic to both. After a few weeks, it’s clear that the short version is converting more customers and that the marketer was correct.
- A business sends weekly marketing emails to its subscriber lists. The goals are high open and click-through rates. The company wants to find an optimal time to send emails for the best open rates. It sends half of the emails Monday morning and a half Tuesday morning. The Tuesday morning emails score higher open rates.
Why Use A/B Testing for Marketing?
As you can see from the examples above, A/B testing helps in a wide variety of scenarios. Within the marketing arena, split testing is useful for:
- Validating assumptions. Gut reactions might tell you people should be excited with words like “free” or “BOGO” in an email subject line, but is that true? Data lets you answer with certainty, and A/B testing is how you get that data.
- Customizing approaches for audience sectors. For example, a younger audience might respond more favorably to certain ads or content on specific social platforms, while older audiences prefer a more traditional approach. But it’s a bad idea to assume these things. Instead, use split testing to validate various approaches with sectors of your audience to find what works best for each one.
- Narrowing your marketing strategy. Marketing resources aren’t infinite. Whether you’re a midsize business with a relatively small budget or a corporate giant with a big budget, the budget does exist. And that means you may have to say no to certain marketing efforts. A/B testing helps you discover which strategies are the most effective so you can concentrate your resources appropriately.
Some of the benefits of strong split testing processes can include:
- More strategic branding. You can figure out exactly what your audience wants and position your brand to provide it. If you’re trying to serve everyone, you’re not going to be as effective as when you home in on those very specific consumer or client problems you can solve.
- Reduced cost of acquisition. Continuous A/B testing helps you chart a map of the least costly way to obtain customers. Once that path is charted, you can use it repeatedly to get the same results.
- Improved KPIs. Split testing can help you improve important marketing KPIs, including email open rates, link, and ad click-through rates, and conversions.
What Can You Split Test for Marketing?
You can use A/B testing on almost any marketing effort if you have reliable access to the right data. Check out some of the ways you can utilize A/B testing to drive better marketing performance:
- Email subject lines. Run tests to discover whether your audience prefers short or long subject lines, how emojis or certain words change open rates, or whether asking a question drives better performance.
- Blog or landing page titles. Do the same with titles on your web content. Does your audience respond to a sense of urgency, humorous titles, or questions? Does including a number in the title make a difference? These are just some of the questions you can answer with split testing.
- Content format or layout. Try out various amounts of white space, compare the performance of video and text content, or test what happens when you include an infographic. You might even split test to see if content in paragraphs performs better than content in bulleted lists.
- Calls to action. Run tests to determine where CTAs should go in the email or on a web page, whether your audience responds better to buttons or text links, and what type of language seems best at persuading people to act.
- Ad text. You can split-test PPC, display, and search ads. Because there’s very little room for text in ads, the words you use matter a lot. You usually have time for a single feature, point, or offer. A/B testing lets you determine which words and offers garner the most interest.
- Ad visuals. Whether you’re working with videos or images, you can do the same with ad visuals to find something that resonates best with your audience.
- Timing of emails. Use split testing to narrow down the dates and times for optimal email marketing performance. While you can find plenty online about the best time to send emails, you need to know the best time to send emails to your audience, and your data is the only thing that can tell you that.
- Ads placement. Run tests to determine if you get better results with certain ad networks or advertising on specific sites or social media platforms.
- Length of content. Should your email marketing message be 100 words or 300? Does your audience want to see short-and-sweet blog posts or pillar pages that run into thousands of words? Split testing can tell you.
- Keyword use. Whether you’re including keywords in your web page content or using them to bid on search ads, A/B testing can help you understand which keywords are best to target, given your unique audience and goals.
- Style of writing. Even the way you phrase your content can be tested. Use it to answer questions such as whether funny, serious, conversational, or formal writing will resonate best with various sectors of your audience?
Tips for Successful A/B Testing
Either split testing starts to sound like a promising endeavor—or you want to know how to do it better—just follow the tips below.
1. Ensure You Have Access to Your Data
When it comes to making marketing decisions based on data, your data is king. You can certainly research benchmarks or see what competitors are doing, and there’s value in understanding what’s working in your niche. But ultimately, the questions you’re answering with A/B testing are about your audience, not someone else’s.
What works on average in the industry may not work best for your audience. For example, imagine a medical billing company specializing in physical therapy clinics products. The type of marketing that works for billing solutions designed for hospitals or even physician practices may not be the best marketing for this specialized business.
How do you get access to your data? You can gather it via website tools, such as Google Analytics, and from basic reports in email marketing solutions. Or you can invest in a tool like Sugar Market, which integrates with a wide range of CRM solutions to ensure you get access to customer and marketing data in a single location.
2. Use Automated Tools When Possible
Manual A/B testing isn’t impossible, but it’s tedious and improbable if you want to make a big impact on marketing. For example, imagine testing different ad copy. You would have to manually run and track ad campaigns and compare the data to determine what was working best. Then, it takes manual work to switch over to the better option.
Here’s an example of how automation with a solution like Sugar Market can make A/B testing easier:
- You decide to test two versions of an email subject line.
- Sugar Market sends version A to around 30% of your list. It sends version B to another 30%.
- Sugar Market tracks analytics such as open rates for those emails. When enough data is in to determine which version worked better, it sends that version to the remaining people on the list.
This is just one of the ways automation can save you time and effort and make A/B testing faster, so you’re acting on the best solutions as soon as possible.
3. Create a Plan for Complex Testing
When you want to find the best option between many, create a testing plan for the best results. You can think of this as a tournament bracket: You have numerous players and want to know who is best.
Let’s use the example of when to send marketing emails to illustrate this point. Perhaps you know you want to send the emails on a weekday, but you’re unsure which day and whether mid-morning or mid-afternoon is best.
That’s ten potential times. You might create a tournament-style bracket to narrow them down:
- Monday early vs. Monday late = Winner 1
- Tuesday early vs. Tuesday late = Winner 2
- Wednesday early vs. Wednesday late = Winner 3
- Thursday early vs. Thursday late = Winner 4
- Friday early vs. Friday late = Winner 5
Then you run even more A/B tests:
- Winner 1 vs. Winner 2 = Winner A
- Winner 3 vs. Winner 4 = Winner B
- Winner 5 vs. Winner A = Winner C
- Winner B vs. Winner C = Final winner
The final winner is the optimal time to send your emails for now.
4. Define Realistic Success Parameters
Be realistic with your goals for A/B testing. Ultimately, any improvement is moving in the right direction, and you’re unlikely to see a result like double the open rate from a single change.
Instead, use continual A/B testing to tweak marketing efforts and move performance in the direction of your goal.
It’s also worth noting that what supports success for you today may not work as well tomorrow. Keeping an eye on your marketing analytics and testing efforts regularly helps you hit what can be a moving target.
5. Be Careful When Testing Multiple Elements
It’s not impossible to test multiple elements simultaneously, such as the words in a subject line and the position of a CTA in the email body. But successfully concluding such tests does take expertise in statistics. Even then, it’s easier to arrive at a wrong conclusion when you’re testing multiple elements simultaneously. Whenever possible, structure A/B testing to validate one element at a time.
Find out more about Sugar Market to discover ways to boost marketing performance while reducing labor costs and unnecessary spending on efforts that don’t work. Book a demo today.
