Logo

Incrementality and holdout tests

Marketing Ideas Finder › Incrementality and holdout tests

Incrementality and holdout tests

In short: Run geo holdouts or platform lift tests to measure what a channel actually adds, beyond what would have happened anyway. It typically needs a budget of £5k+ a month, about 3 hours a week, and results usually show within 1–3 months. KPI to track: Incremental CPA vs reported CPA. It is a strong fit for PPC / paid media, Growth / performance marketer, Marketing manager and others.

At a glance
Budget£5k+/mo
Time neededAbout 3 hours a week
Time to results1–3 months
Skill levelAdvanced
KPI to trackIncremental CPA vs reported CPA
ChannelsStrategy & leadership, Analytics & research, Paid search, Paid social
Main goalsInsight & measurement, Sales
Funnel stageConversion
Business typesB2C, Ecommerce / DTC, SaaS, B2B
Company stageSMB, Enterprise

Why it works

Platform-reported conversions often include sales that would have happened without the ads. Holdouts show the true lift.

How to do it: 5 steps

  1. Pick a channel with significant spend
  2. Design a geo or audience holdout with a control group
  3. Run it long enough to read results
  4. Compare incremental cost per acquisition with platform-reported numbers
  5. Reallocate budget and repeat for other channels

How to measure it

Track Incremental CPA vs reported CPA. Typical time to see results: 1–3 months. Record a baseline before you start and compare against it before you scale up.

Who it suits

Best fit for: PPC / paid media, Growth / performance marketer, Marketing manager, Senior digital marketer / Head of, CMO / marketing director.

Also useful for: Product marketer, B2B marketer, Marketing ops.

Business types: B2C, Ecommerce / DTC, SaaS, B2B. Stage: SMB, Enterprise.

Do this with AI

Copy this prompt into ChatGPT, Claude, Gemini or Grok, replace the words in [brackets] with your details, and you will get a tailored plan for this idea.

Act as an experienced marketing director. I want to put this marketing idea into practice: "Incrementality and holdout tests".
What it means: Run geo holdouts or platform lift tests to measure what a channel actually adds, beyond what would have happened anyway.

About me (replace the brackets):
- Business: [your business], which sells [what you sell]
- Audience: [audience]
- Goal for the next 90 days: [goal]
- Budget: [monthly budget] (this idea usually needs £5k+/mo)
- Time: [hours per week] (this idea usually takes about 3 hours a week)
- Current numbers: [current traffic, leads, sales or list size]

Work through these steps and give me concrete, specific output for each one, tailored to my business:
1. Pick a channel with significant spend
2. Design a geo or audience holdout with a control group
3. Run it long enough to read results
4. Compare incremental cost per acquisition with platform-reported numbers
5. Reallocate budget and repeat for other channels

Then give me:
1. A one-page plan with objectives, owners, timeline, budget and decision points
2. How to measure success: track "Incremental CPA vs reported CPA". Suggest realistic targets for 30, 60 and 90 days based on my current numbers, and say what to do if results are below target.
3. The three most common mistakes with this tactic and how to avoid them.

If anything important is missing, ask me up to 3 short questions first. Be practical, use plain English, and do not invent statistics, case studies or quotes.

Tools and help on Digital Marketing Supermarket

Related marketing ideas

Plan it with the Marketing Ideas Finder. Filter 401 ideas by budget, time, role and channel, and build a 30/60/90-day plan.

Added 17 Nov 2026 · Updated 18 Nov 2026 · Part of the Marketing Ideas Finder by Digital Marketing Supermarket.