As companies are finalizing budgets for next year and marketers are fine-tuning their plans, it’s a popular time to look back at what worked and what didn’t. It’s also a critical opportunity to revisit your organizational strategy, realign your KPIs, and design a testing system that gives you actionable insights.
“We do A/B testing – so we’re covered, right?” Not necessarily. While A/B testing can be a really valuable tool for evaluating your content, it doesn’t tell you nearly as much about your targets. And this can be a huge miss for marketers because let’s face it: your color scheme doesn’t matter if your target audience is wrong to begin with. So how do you go beyond A/B testing to maximize your budget and see some wins?
It fundamentally begins with understanding your business strategy. Too often, I see marketers who are unaware of the organization’s strategic priorities and therefore, they’re chasing too many goals at once. “We want to test x, y, and z all at the same time within a test.” That’s not often the best approach because you can get confusing results when you have multiple variables like that.
It’s important to know, from an overall business standpoint, what is considered most critical. Is it to grow customers? Or is it to cut costs and operate more efficiently? Most companies will say both. Is acquisition more important than retention? “We want both.” It’s common to have competing goals within an organization, which is why it’s so important to establish priorities. Knowing this has a real impact on what comes next.
As long as you can define your primary objective, you can identify what needs to be tested. If the goal is to grow customers, for example, we need to figure out what we need to test to achieve better growth. Maybe we test high-yield offers against certain segments to understand if they produced lift and if they were, indeed, high-yield. I’ve seen this done well in direct mail, where the results are relatively easy to measure and read because it’s direct-response. But that also can’t be a one-shot deal.
A mistake people often make is not testing long enough, consistently over time, or at a level that is statistically significant to yield meaningful results. Sometimes, this is a budget-based decision – for example, you don’t want to give up your control group out of concern for what you might lose by letting them rest – but ultimately, you have to factor in that cost as part of establishing overall efficacy. You have to commit to testing over a long enough period of time that you can rule out external factors like seasonality and get meaningful, readable results. In addition, customer attitudes change over time, and a limited window may provide a skewed picture.
Any time you’re testing something new, the risk of “failure” is real. And there are budget implications associated with that. Spending time and resources developing a test when you could have been doing something else is opportunity cost. Maybe you spent time on an acquisition test when you could have been resourcing retention. But you have to test to know – every campaign provides an opportunity to build knowledge.
Testing involves versioning and in digital testing, the risk is relatively low; in other cases, there’s more at stake. That’s why aligning your marketing goals with the organization’s strategic priorities is so critical. Every test, successful or otherwise, is a step forward - so make sure you’re stepping toward the right thing with the right kind of support.
To successfully execute testing (and reading and adjusting to results) on an ongoing basis, there needs to be organizational – and cross-department/cross-functional – alignment as well as marketing-specific support. Whatever that function is – an in-house marketing team, an agency (or agencies) – it has to be able to recognize and support that testing requires multiple versions, whether that’s creative (A/B testing, for example) or logistics (like having multiple toll free numbers into your call center).
It really comes down to defining your learning objective and having a process for testing, then reading and understanding your results, and moving into production. Once you’re in production, continue testing and if things need to change, be ready with a backup plan. In other words, remain focused and agile and manage your budget and organizational constraints to work smarter. If you’re willing to take some calculated risks, the results can be really impactful.
