Answers to your questions from the Kochava Foundry webinar
Grant Simmons, VP of Kochava Foundry, recently hosted the webinar Measuring Incrementality & Lift, where he unpacked lift measurement best practices, common pitfalls with hold-out groups, emerging methodologies for incrementality testing, and Kochava solutions for incremental lift measurement. The Foundry team gathered some of the most engaging audience questions Grant answered during the webinar to elaborate on in further detail.
Check out the full webinar on demand here.
How exactly does Kochava Foundry measure incrementality & lift for content partners that run ads year round with no dark periods?
Foundry employs various measurement techniques to assess incrementality and lift for content partners with ongoing, year-round ad campaigns. While MediaLift™ is primarily designed for discrete campaign measurement, Foundry can utilize modeling techniques to understand the ongoing lift of always-on media.
By leveraging regression discontinuity, Foundry can analyze the impact of continuous advertising efforts and identify the incremental contribution of specific media partners. Additionally, AIM (Always-On Incremental Measurement) can be employed to measure the contribution of each channel continuously at the network, publisher, and campaign level. This enables content partners to gain insights into the effectiveness of their ongoing ad campaigns and make data-driven decisions to optimize their media mix.
Can Kochava MediaLift be used for an advertiser that is not using Kochava as their mobile measurement partner (MMP)?
Yes, Kochava MediaLift can be utilized by advertisers who don’t have Kochava as their MMP. MediaLift is designed to work with standardized ad signals and conversion signals, making it platform-agnostic. This means that even if an advertiser uses a different MMP, MediaLift can still analyze the ad and conversion data to measure incrementality and lift. By leveraging MediaLift, advertisers can gain valuable insights into the incremental impact of their campaigns, regardless of their chosen MMP.
What are some best practices for running incrementality tests while minimizing negative impact to my business?
- Define clear objectives: Clearly define what you want to measure and the specific metrics you are targeting. This guides the design of your test and ensures that you are capturing meaningful data.
- Consider opportunity and hard costs: Holdouts, or not marketing to a portion of your audience, can be costly and result in missed opportunities. Consider the potential impact on your business and weigh the costs against the benefits of the test.
- Determine timing and duration: Choose the appropriate timing and duration for your test to capture meaningful data without disrupting your ongoing campaigns. Consider factors such as seasonality, campaign duration, and audience behavior to ensure accurate results.
- Explore modeled options: Modeled approaches, such as synthetic control groups or machine learning models, can provide reliable results while mitigating the opportunity cost of holdouts. These approaches use historical data and statistical modeling to estimate the incremental impact of your campaigns.
Would you recommend measuring incrementality on your own or with a third-party company—or somewhere in-between?
I encourage brands to develop their own solutions. After all, it is YOUR money, and the measurement tooling you run must be up to your standards.
While it is possible to measure incrementality on your own, and Foundry has helped brands get there, it is often beneficial to work with a third-party company that specializes in incrementality measurement. Here’s why:
- Expertise and tools: Third-party companies have expertise in designing and executing incrementality tests. They can access advanced tools and methodologies that provide more accurate and reliable results. Their experience in analyzing large datasets and understanding statistical models ensures that measurements are conducted effectively.
- Unbiased and objective insights: Third-party companies provide unbiased and objective insights into the incremental impact of your campaigns. They are not influenced by internal biases or vested interests, allowing for a more impartial evaluation of your marketing efforts.
- Scalability and efficiency: Third-party companies have established processes and infrastructure in place to handle large-scale incrementality measurement. They can efficiently analyze and interpret the data, providing timely and actionable insights.
That said, it is important to find the right balance between in-house measurement capabilities and third-party expertise. Some advertisers may build internal measurement capabilities while leveraging third-party support for more complex analyses or to validate their findings.
Since leaders like to look at annualized numbers, but we don’t know decay, how can you scale the incremental numbers over a year? Any best practices?
- Calibrate attribution models: Data from incrementality tests is used to refine and calibrate attribution models. By understanding the incremental impact of different channels and tactics, you can adjust the attribution weights assigned to each touchpoint in the customer journey. This ensures that the attribution model accurately reflects the true contribution of each channel and tactic.
- Consider seasonality: Take into account any seasonal variations in your industry or market. Adjust the scaling of incremental numbers based on historical patterns during specific periods.
- Allocate budget: Use insights from incrementality tests to allocate budget toward channels and tactics that drive the most efficient incremental cost per acquisition (iCPA). By identifying channels and tactics that generate the highest lift and incremental conversions, you can prioritize budget allocation accordingly. This helps optimize marketing spend and maximize return on investment.
- Re-measure optimizations: Continuously re-measure the impact of optimizations based on the results of incrementality tests. By implementing changes to your campaigns, such as adjusting targeting parameters, creative elements, or bidding strategies, you can evaluate how these optimizations contribute to greater incremental contribution. This iterative process enables you to refine your strategies and make data-driven decisions to drive incremental growth.
Generally speaking, do you think online brands with smaller brand awareness can put more value or trust in incremental lift tests in some ways? Or do you find that even smaller brands can run into the same issues?
- Smaller brands may be in a better spot vs. big brands in that the smaller brands have less brand equity. So theoretically, ad spend should provide more of a pop because ad media is the only way some folks will come to know a new brand.
- Incremental lift tests can help smaller brands identify the specific channels, tactics, or campaigns that are driving incremental results and optimize their marketing strategies accordingly. By measuring the lift in conversions or actions compared to a control group, smaller brands can gain insights into the true impact of their advertising efforts and make data-driven decisions to allocate resources effectively.
- However, it is important to note that smaller brands may still encounter challenges similar to larger brands when conducting incremental lift tests, such as ensuring proper test design, data quality, and statistical significance (i.e., amount of data). It is crucial for all brands, regardless of size, to plan and execute their incrementality tests carefully to obtain reliable and actionable insights.
Do holdout biases exist with geo-based holdouts, or is this exclusive to audience-based splits?
This is likely so, but geo holdouts can be a useful tool. Assuming that two markets move in concert, if one is treated with media and the other goes dark, the marketing effect lift may be understood as the performance of the two markets in direct comparison. Note that this usually takes an inordinately large amount of spend in the target market and having to go dark in the control market, which isn’t how you would actually run the campaign, so the results may not actually reflect reality.
While geo-based holdouts can be a useful tool for comparing the performance of different markets, it is important to consider factors such as market dynamics and the significant amount of spend needed in the target market.
Conducting holdout tests in a way that accurately reflects real-world campaign execution can be challenging. Going dark in the control market may not truly replicate how the campaign would actually be run, which can introduce biases and affect the validity of the results obtained from the holdout test.
What are your thoughts on a cross-screen campaign? Using an always-on approach and multi-touch attribution, we would want to tell the power of our multiple products.
- A cross-screen campaign with an always-on approach and multi-touch attribution can be a highly effective strategy to showcase the power of multiple products. By maintaining a consistent presence across various screens, you can engage with your target audience at different stages of their customer journey. This allows for a more holistic and integrated marketing strategy—maximizing the exposure and impact of your multiple products.
- Multi-touch attribution enables you to understand the contribution of each touchpoint in the customer journey and measure the incremental value generated by each product. It provides insights into how different screens and touchpoints work together to drive conversions or actions, allowing you to optimize your campaign and allocate resources effectively. To execute a successful cross-screen campaign, it is important to have a robust measurement framework in place, including proper tracking, attribution mechanisms, and advanced analytics tools. Continuous monitoring and analysis of campaign performance will help ensure its effectiveness.
Got more questions on measuring incrementality and lift?
Are you looking for expert consultation on your incrementality testing strategy? Need help measuring the lift of your campaigns? Get in touch with the Kochava Foundry team for an expert consultation.
