Written in partnership with Measured, a leading provider of incrementality testing and media optimization solutions.
Unlock Revenue with Incrementality Measurement and Scale Testing
Every brand and marketer strives to get the most out of their media investments, and uncovering hidden pockets of incremental revenue is an exciting opportunity. Traditional measurement methods such as, Marketing Mix Modeling (MMM) or Multi-Touch Attribution (MTA) have their limitations. With that said, there is a better way to scale media with greater accuracy and precision. Incrementality testing has emerged as a more effective methodology, enabling brands to unlock previously untapped business potential. At Transparent Partners, we work with best-in-class measurement partners like Measured to help brands achieve exceptional growth by unlocking untapped business potential.
Brands Look to Scale for a Number of Reasons
Budgets are Frozen but Still Need to be Optimized: Most marketers entered 2023 amid economic uncertainty and with budgets that are more or less flat from 2022. But the need to optimize remains. Through measurement, marketers can better understand channel performance and then harness scale testing in order to shift their media mix. The result can potentially unlock incremental revenue.
Marketers Have Gotten Too Comfortable: A recent HBR article pointedly noted: “Inertia is powerful. Every organization has legacy spend patterns or processes that aren’t effective or efficient. This doesn’t have to mean cutting overall investment; it could involve simply redeploying resources to their highest and best use.” Now is the time to take a hard look at channel performance and prove the value of media investment.
New Channels and Publishers Continue to Emerge: Channels and publishers are continually emerging. It may be difficult to believe, but Facebook earned its first marketing dollar just over 15 years ago. Over-the-Top (OTT) (e.g., CTV) really took off within the last ten. TikTok has only been relevant the last few. New channels result in new platforms, technology, and measurement challenges that marketers need to figure out. Additionally, marketers also need to identify best practices for scaling up media spend in order to reach financial goals.
Brands (Usually) Look to Maximize on Opportunities: When times are good and incremental budget becomes available, brands must quickly identify where to best spend their next media dollar to maximize performance. This includes developing a framework for determining which channels are most profitable. Additionally, marketers need to figure out how much to invest in them, in order to achieve their business goals. When looking to scale spend (up OR down), selecting the appropriate methodology is critical.
Selecting the Right Measurement Methodologies to Scale Test
When looking to evaluate media’s impact on conversions, there are generally six categories of methodologies that marketers employ. (A deeper look at all six methodologies can be found here and how to navigate disparate measurement outputs can be found here.)
Every brand has a different mix of these six at their disposal. Enterprise-level brands typically have some form of MMM (whether internal or 3rd-party), single-channel MTA (e.g., Google), and potentially (hopefully) some incrementality testing. Smaller brands may rely entirely on publisher-reported data and/or MTA, with some in-house MMM modeling. Regardless of size, when looking to understand and estimate a channel’s ability scale, Incrementality testing is the best positioned methodology.
What is Incrementality testing?
Incrementality in marketing is the lift or increase in the desired outcomes (conversions) caused by a specific marketing activity. Measuring incrementality identifies the conversions that are above and beyond what would have happened anyway if the marketing activity being measured had not taken place.
Incrementality testing can reveal the true contribution that an ad channel, campaign, or tactic makes to desired outcomes, such as sales. Unlike MTA and MMM, incrementality measurement identifies causation to answer the question, “which conversions were caused by that ad campaign?” The insights acquired through incrementality testing are based on data evidence from scientifically sound test and control experiments. Using incrementality measurement, brands can identify areas to eliminate wasted budget and reveal opportunities to scale and grow.
Incrementality testing methodologies can vary depending on a number of factors. This includes, publisher technological capabilities, ability to track at the user level, and characteristics of the channels themselves. Legislation-driven (as well as self-imposed) tracking deprecation has made methodologies that require user-level tracking largely obsolete. This is true for “Audience Split” testing, as an example.“Geo-Matched Market Testing” (“Geo”) is now the best and most prominent methodology currently being deployed in the market.
Why is Geo best positioned methodology for scale testing? 1) It features flexibility in test design to directly answer specific business questions. 2) It is a near-universal approach because it can be deployed on any channel with geo-level targeting and metrics. 3) It enables marketers to explore well beyond the bounds of other methodologies. 4) It offers the capability to simultaneously test multiple spend levels, which is central to the topic here.
When using geo-experiments to scale test, media campaigns will be segmented into multiple “geos” (e.g., DMAs, states, zip codes) and funded at significantly different levels of spend. These geo segments represent a test “cell” that will be compared against a carefully selected “control” geo. This approach is also cost effective given that the scaled up spend only occurs in a few (carefully selected) markets (also, many markets will have zero spend).
Upon completion of the test, a “diminishing returns” curve should emerge that identifies saturation points and would allow marketers to estimate the level of spend necessary to hit, say, an incremental return on ad spend (iROAS) of $5.00 (as shown in Example 1) which looks to be between $2M – $2.5M, or to hit, say, an incremental Revenue (iRev) target of $10M (as shown in Example 2) which appears to be slightly less than $2M.
How to Scale Media Intelligently
There are a handful of best practices to help you implement incrementality and scale testing to be most effective.
Make a plan. Set yourself up for success by developing a learning agenda that is shared between the brand, measurement partner, agency partners, and any other relevant stakeholders. A well defined agenda will:
- Prioritize the business questions you want answered, so you can identify the most appropriate test designs for answering them.
- Support creation of a testing roadmap. Brands typically start testing incrementality on the channels and tactics that comprise the largest share of marketing spend. From there, they run scale tests to determine how much more to invest in the most profitable strategies.
- Enable brands to track test results and evaluate the impact of the decisions they drive.
Commit to Decisioning. Measurement without decisioning is just expensive reporting. Establishing a decisioning framework prior to conducting a test encourages stakeholders to commit to a particular decision based on the outcomes of the test. Every result should inform what you decide to do next.
Work with testing experts. Successfully executing incrementality testing through geo-matched market testing, can become very complex. Market selection requires the application of advanced data science to ensure test cells are statistically significant. While a handful of publisher platforms offer incrementality testing, there are a variety of reasons why marketers should be wary of taking this path.
Measuring incrementality with independent geo-testing is the only way to confirm whether platform reporting is accurate. And most often, it is not. Incrementality is a common currency that enables marketers to compare performance across multiple channels and tactics. It also won’t break when new tracking restrictions and privacy regulations are rolled out across the industry.
Measured Use Cases – Incrementality and Scale Testing in Action
Below are a few results from actual scale tests, that Measured has deployed for ecommerce brands.
- Challenge: Honeylove wanted to know how best to allocate acquisition budget across channels. They “sensed” their Pinterest awareness campaigns were performing better than what the platform was reporting and wanted to learn how much more to invest in the channel.
- Solution: 30-day scale test at 3X in four states that mirrored nationwide target market.
- Results: The ROAS of Pinterest awareness campaigns proved to be 212% higher than what was being reported by the platform. The brand confidently doubled its investment in awareness campaigns.
- Challenge: Google Analytics frequently attributes more conversions to Search than any other tactic. The marketing team wanted to know how to allocate budget between Google Search and Shopping tactics for maximum ROAS.
- Solution: State level geo-matched market experiment to measure the true incrementality of their Google Brand Search and Google Shopping campaigns.
- Results: Actual sales volume driven by Brand Search was just 6% of what Google claimed, while the sales volume contributed by Shopping was 115% of that reported by Google. By divesting from Search into more profitable tactics, the brand increased ROAS by 13%.
- Challenge: Shinola relies heavily on Facebook for customer prospecting but after Apple rolled out tracking restrictions with iOS.14.5, Facebook-reported conversions were way down. Shinola needed to know what was really happening so they could make decisions about where to increase investment.
- Solution: Zip-level geo testing to reveal the true value of Facebook awareness campaigns at various levels.
- Results: The test revealed that Facebook was underreporting awareness results by 413%.(!!) Shinola then ran scale tests to determine how much more they could spend on the tactic while maintaining target ROAS.
How Measured and Transparent Help You Win
Addressing tough questions and significantly shifting budgets is no easy task and there is no measurement “silver bullet.” An oft-cited HBR article notes: “Companies that take a scalpel rather than a cleaver to [their] marketing budget, and nimbly adjust strategies, tactics, and product offerings in response to shifting demand are more likely than others to flourish.” Transparent Partners helps brands to bring together the best methodologies, tools, and partners to take a “scalpel” approach, which is critical to succeeding at modern marketing measurement and decisioning.
Transparent works closely with best-in-class measurement partners such as Measured, who are leading the pack in innovation around incrementality testing. The Measured Platform was used to optimize more than $5 BILLION in media spend in 2022 alone. Transparent helps brands evaluate and contextualize multiple measurement outputs, facilitate any needed cross-functional change management, and deploy cutting-edge incrementality tests to identify and eliminate marketing waste and uncover opportunities to unlock incremental revenue.
Transparent Partners
