How a market test works

If you have always answered market questions with surveys, panels or focus groups, a market test can feel strange at first. Nobody is recruited. Nobody is asked anything. Most of the people in it never respond at all. And yet it tells you, with a confidence level attached, which of your ideas your market wants most. This guide explains how that works, why the top strategy consultancies have relied on it for over a decade, and how Heatseeker runs one in a week.

Where the method comes from

Market testing is not new. For well over a decade BCG, McKinsey and Bain have used it to de-risk launches for their clients, under names like smoke test, fake-door test and landing-page test. The idea is always the same: put the proposition in front of real buyers as if it were real, and count who steps forward. Stated preference, what people say they would do, predicts what they actually buy only about 20 to 30 percent of the time. Behaviour in the buying moment is the better guide, and a market test is the cheapest way to observe it before anything is built or bought.

What kept it a consultancy tool was the overhead. One test needed a team: someone to design the variants, someone to build the pages, a media buyer, an analyst to read the result. Worth it for a big bet, and for nothing else.

Heatseeker turns the same method into something one marketer runs in an afternoon. It designs the ads, sets up the experiment, holds every variable constant, and reads the result with the same decision statistics Netflix, Microsoft and Optimizely use to run experiments at scale. The rest of this guide is what happens underneath.

The five ideas that make it work

A market test rests on five ideas. Each answers a question people who come from survey research tend to ask.

1. One variable

In a message or visual test, every ad is identical except for the one thing you are testing. In a message test the image, audience, budget, schedule and placement are all the same, and only the words change. A Full Creative Test is the deliberate exception: it changes the whole ad, and its winner is a direction rather than a single element. Below, a quick-service restaurant chain tests which job to lead with at lunchtime.

Holding everything else constant goes further than it would in a normal campaign. Each cell runs exactly one creative: no carousels, no rotation, no in-flight variations. The platform's own optimisation, which exists to expand an audience and move budget toward whatever is winning, is held off, so the platform cannot favour one cell over another while the test is live. Objective, bid strategy, budget, schedule and targeting are identical in every cell. The only differences are the ad and the random slice of the audience that sees it.

2. Granular targeting

A market test runs with the people you already plan around. On Meta and TikTok that is life stage, location and interests. On LinkedIn it is job title, function, seniority, industry and company size. Nobody is recruited and nobody is paid. The people in the test are simply your segment, reached where they already spend their time, and none of them know a test is running.

That precision is what makes the answer yours. A survey panel gives you people roughly like your customers who are willing to answer questions. A market test gives you your actual market. And because each segment can be tested on its own, you learn which message wins with whom, not just on average.

3. Random split

This is the trick that makes everything else possible. Once the audience is defined, the platform decides who sees which ad, at random. Each person lands in exactly one cell. Each cell gets an equal share of the audience.

Example / One segment, split at random, one ad per cell

1 / Your segment

Everyone the platform can reach who matches your targeting. Nobody is recruited and nobody knows a test is running.

2 / Random split

The platform assigns each person to exactly one cell, by chance.

Equal shares. Nobody sees two variants. Nothing is weighted or matched afterwards.

3 / Equal cells, one ad each

• Cell A · 25% of the segmentResponded
  “Lunch in three minutes.”

  1.6%
• Cell B · 25% of the segmentResponded
  “Made when you order.”

  1.5%
• Cell C · 25% of the segmentResponded
  “A proper lunch under ten dollars.”

  1.3%
• Cell D · 25% of the segmentResponded
  “Your lunch, your way.”

  0.8%

The cells are alike in every way that matters, so a different response rate comes down to the ad or to chance, and the Signal rating accounts for chance.

Because the split is random, the cells are alike in every way that matters: the same mix of ages, places, moods, times of day and devices, on average. Nothing is weighted, matched or modelled afterwards. The only thing that differs between cells is the ad, so a difference in response comes down to the ad or to chance. Chance is what the Signal rating accounts for.

How the split is enforced differs a little by platform: Heatseeker runs the test inside the platform's own experimentation framework where one fits, and approximates the split by tuning the delivery settings where it does not.

4. Scale

A market test is quantitative in a way few research methods are. Each variant is seen by tens of thousands to hundreds of thousands of people in a typical week. A focus group hears from eight people. A survey hears from a few hundred who agreed to answer. A market test observes everyone in the segment the platform can reach, without asking them anything.

5. Revealed preference

Finally, a market test measures what people do rather than what they say. Every signal the platform exposes counts: engagements, clicks, form opens and leads, weighted by how much effort they take. A click counts more than a like, a form submit more than a click. Everything is divided by the number of people who saw the ad, so reaching more people earns no credit. The winner is the message, not the budget.

A click is not a purchase, and the method does not claim it is. What it establishes is narrower and more useful: shown to the same people, at the same time, for the same money, the speed message made twice as many of them act as the choice message did. That is the decision in front of you before a campaign, a launch or a roadmap: not how many will buy, but which idea deserves the budget. Whether the pull turns into sales is what your conversion data confirms once you scale the winner.

What about the people who didn't click?

This is the question that trips up almost everyone who comes to market testing from surveys, and it deserves a direct answer.

In a survey, a person who does not take part is a gap in the data. In a market test, a person who sees the ad and scrolls past is a data point. They saw the message and took no action, and that is recorded. They are counted, in the denominator, every time.

The random split is what makes this work. Every cell has the same share of people who were never going to act: not hungry, not shopping, not in the market, not paying attention. That share is the same in cell A as in cell D, because nobody chose their cell. So when cell A responds at 1.6% and cell D at 0.8%, the uninterested majority has cancelled out, and the difference that remains is the message.

Three things follow from this.

• A low response rate is not a weak result. Most people do not respond to most ads, and the test does not need them to. It needs the rates to differ between cells. With tens of thousands of people in each cell, a difference of a fraction of a percent is a large and reliable gap in pull.
• You cannot ask the uninterested, and you do not have to. A survey hears from the people willing to answer, and what it hears is intention. A market test lets everyone vote with their attention, including the ones who vote no.
• This is what the market will do anyway. A campaign is shown to everyone in the segment and most of them scroll past. A market test measures exactly that behaviour, on the same platform, with the same people, before you spend.

Why a week is enough

A conventional A/B test optimised on conversions can take three or four weeks of meaningful budget to call a winner. A Heatseeker test reaches a decision-grade read in about a week, on the same platforms with the same audiences. The speed is not a shortcut. It comes from four design choices that compound.

In practice, seven days is the default and covers a full weekday and weekend cycle. Three to five days is achievable when the audience is broad and the signal strong. Smaller audiences may need seven to nine. A genuinely close test can be extended until the read firms up or the budget cap is reached, and at that point a result of no clear winner is itself a finding: the evidence cannot tell these variants apart, and the decision can be made on other grounds.

How a market test compares

Each research method has a job. The matrix shows what a market test is for, and what it is not.

It is also not A/B testing. The two use similar machinery and answer different questions. A/B testing tunes something already live: a headline, a layout, a button on an existing page or funnel, with spend already committed. A market test decides what should go live in the first place: the proposition, the audience, the message, the concept, before the product or the media budget exists. On Meta, both can run inside the same experiments framework. The difference is that Heatseeker locks the dials Ads Manager leaves open, and reads the result with a different engine: a weighted score across every engagement signal instead of one conversion event, a ranking by how much each variant gives up instead of a pass or fail, and a stop the moment the answer is clear instead of a fixed window.

And it is not a replacement for everything. Surveys and interviews remain the tool for why, for pricing research that needs a stated trade-off, and for questions your market cannot answer with a click. Most teams run the market test first to find the direction, then use the conversation to understand it.

What a market test tells you

A market test gives you a ranking of your options within a segment, and a signal that says how far to trust it. To make the ranking mean something, Heatseeker reads it against three yardsticks.

• A baseline inside the test. Run the message you use today as one of the variants. Every challenger is then measured against what you actually do, under identical conditions, and if nothing beats it you have learned that for the price of one test.
• Your own performance benchmarks. With your ad account connected, Heatseeker AI can set the test's response rates next to your campaigns' rates for the same audience. See Enable Meta ad performance data and Enable LinkedIn ad performance data.
• General market benchmarks. Typical rates for the platform and the category, so that a response rate of around one percent reads as the normal number it is, rather than as a disappointment.

Read the comparison as context, not a forecast. Test ads are deliberately different from one another and run without the platform's optimisation, so they are not a preview of campaign performance. They are a fair fight between ideas.

Will it touch our live campaigns?

Performance marketers ask this first. In practice, no, for three reasons.

1. Randomisation absorbs overlap. Any overlap between the test audience and your live campaigns affects every cell equally, because people are split at random inside the test audience. The comparison between cells stays clean.
2. Test budgets are small by design. Per-variant spend is a fraction of an always-on campaign, so the impressions a test takes from your live activity are negligible.
3. The objectives differ. Tests run on engagement and lead objectives, separate from the lower-funnel conversion campaigns where crowding out would actually matter.

Large advertisers run market tests alongside live campaigns continuously. Running in parallel is the norm, not the exception.

What Heatseeker adds to the method

The division of labour is simple. The platform runs the delivery to the target audience. Heatseeker does everything else.

• Designs the experiment. From your question, Heatseeker AI picks the experiment type, drafts the variants to your brand guidelines and in your tone of voice, and pushes back when two variants are too alike to separate. See How an experiment gets designed and run.
• Locks the variables. One creative per cell, equal budgets, identical bid strategy, platform optimisation held off. The dials a normal campaign leaves open are closed before launch.
• Sizes the budget and stops spending early. The budget is a cap, not a target. The test stops the moment the answer is decisive.
• Reads the result as a decision. Every variant on one scale anchored at the winner, a signal that says how conclusive the finding is, and a plain-language recommendation written against the goal you set. See How to read your results.
• Tests several segments at once and breaks results down by demographics on Meta and by firmographics on LinkedIn, so you see which message lands with whom.
• Runs under a lookalike brand when the question should not carry your name yet: an early concept before anything is announced, a bold idea that is off your current strategy, a price point you are not ready to be seen testing, or a benchmark against a competitor's claims.
• Trains your AI customers. Every live result grounds a private, simulated version of your segment that can narrow the next field in minutes. The live run stays the ground truth.

Reading the result

Results open on the winning ad. Every variant is scored on what people did with it, per person who saw it, and shown as a share of the winner. The winner is 100%. A variant at 93% is a real alternative. A variant at 50% is not in the race.

Beside the winner sits the Signal, with four levels describing how conclusive the finding is: Decisive, Strong, Indicative and Inconclusive. Decisive means the evidence clears the decision threshold for the tested choice. Strong means the leader is clear but has not cleared that bar, which is often enough when you are narrowing a field. Indicative is a useful provisional finding. Inconclusive means the evidence cannot yet tell the variants apart, and a completed test can be inconclusive. Behind the rating sit two questions: how much of the best result you keep by backing the winner, 95% or more for Decisive, and how often that pick is safe, nine times in ten or better. A simulation never shows Decisive. The full read, including the by-segment heatmap, is in How to read your results.

Common questions

Is a response rate of one or two percent too low to mean anything?

No. It is normal, and it is the measurement working. The question is never whether most people respond, because most people never respond to any ad. The question is whether one message pulls more of them than another, with the same people under the same conditions. A move from 0.8% to 1.6% is twice the pull, measured across everyone who saw the ad, including the people who scrolled past.

What about the people who saw the ad and did nothing? You cannot know what they think.

You do not need to. They are counted, as people who saw that message and took no action. Because who sees which ad is decided at random, every ad faces the same share of people who were never going to act. Their silence cancels out between cells, and what remains is the difference the message made. A survey has no equivalent: it hears from the people who chose to answer, and it hears what they say rather than what they do.

Will the test contaminate our live campaigns?

In practice, no. Per-variant budgets are a fraction of an always-on campaign, so the test takes very few impressions away from your live activity. On Meta the test is registered as a native experiment, which guarantees that a person sees one variant only. And the test runs on engagement and lead objectives, separate from the lower-funnel conversion campaigns where crowding out would matter.

What if the test audience overlaps with one of our active campaigns?

Overlap is fine. Because people are split at random inside the test audience, every cell experiences the same amount of overlap and the same market conditions. The comparison between cells stays clean.

Why switch off the platform's own optimisation?

The platforms' audience-expansion and budget-shifting features exist to make a single campaign perform better: they widen the audience and move budget toward whatever is winning. In a controlled test that would let the platform tilt the field between cells. Heatseeker switches them off so that the only thing that differs between cells is what you are testing.

How is the random split guaranteed?

Wherever a platform offers an experimentation framework that fits the test, Heatseeker runs the test inside it, so the platform itself assigns each person to one cell and enforces that nobody sees two variants. On Meta that is the same machinery Meta uses for its brand-lift studies. Where a framework does not fit, Heatseeker approximates the split by tuning the delivery settings: every variant runs against the same audience with equal budgets and identical settings, and the platform's own optimisation is switched off so nothing favours one variant. Either way, every variant faces the same audience under the same conditions, which is what keeps the comparison fair.

Does the ad with the bigger budget win?

No, for two reasons. Every variant gets the same budget. And every variant is scored per person who saw it, not on totals, so the winner is the message and not the spend behind it.

Can we include the message we run today?

Yes, and it is a good habit. Your current message runs as one of the variants, so every challenger is measured against what you do now. If nothing beats it, you have learned that cheaply.

How do the numbers compare to our normal campaign results?

With your ad account connected, Heatseeker AI can put the test's response rates next to your own campaign rates for the same audience, and next to typical rates for the category. Test ads are deliberately different from one another and run without optimisation, so read the comparison as context rather than as a forecast of campaign performance.

How long does a test run?

Seven days is the default. That covers a full weekday and weekend cycle and reaches a decision-grade read on most messaging tests. Three to five days is achievable when the audience is broad and the signal is strong. Smaller audiences may need seven to nine. If a test is genuinely close, Heatseeker can extend it until the read firms up or the budget cap is hit, and a completed test can be reported as inconclusive.

How can the result be valid if you stop the test early?

Heatseeker does not run a fixed-sample hypothesis test. It updates a Bayesian decision model as the data arrives, computing how much performance you would give up by acting on what is known so far, and calls the result only when that figure is small enough to be safe. That calculation reads correctly at any point in the run, which is what lets the test stop the moment the answer is there.

Why no correction for testing eight variants at once?

Traditional testing penalises every extra comparison, because each one is another chance for a false positive. Heatseeker does not test variants pairwise. It models the uncertainty of all of them together and ranks each by how much performance it gives up against the best, so an experiment can hold up to ten. The variants share the budget, so a wider field needs more spend or a longer run to keep the same precision.

Aren't clicks just curiosity?

Some are, and the score weights for it: a form submit counts more than a click, a click more than a like. Across tens of thousands of people, under identical conditions, a message that draws more action is a message with more pull. That is not the same as a sale, which is why Heatseeker describes a market test as the fastest way to find the direction worth scaling, and your lower-funnel data as the way to confirm it.

Can competitors see the test?

The ads are real, so anyone in the target audience could see one, the same as any ad you run. If the question should not carry your name yet, such as an early concept or a bold idea that is off your current strategy, run the test under a lookalike brand so the question is not tied to you. The results are private to your workspace.

What if our buyers aren't on Meta, LinkedIn or TikTok?

Almost every audience is reachable on at least one of them, and B2B audiences are usually best reached on LinkedIn by job function, seniority and company size. If your buyers genuinely are not there, a market test is not the right tool, and Heatseeker will say so when you describe the audience.

Glossary

Variant

One of the ads in a test. Variants are identical except for the one thing being tested.

Cell

The random slice of the audience that sees one variant. Equal in size across the test, and nobody belongs to two.

Impression

One person seeing one ad once. Response rates are counted per person who saw the ad, so a person who saw it twice is still one person.

Random split

The platform's assignment of each person to one cell by chance, so the cells are alike before the test starts.

Revealed preference

What people do, as opposed to stated preference, what they say they would do. Market tests measure the first.

Composite score

Heatseeker's weighted count of every engagement signal on an ad, per person who saw it. Stronger actions weigh more.

% of Winner

A variant's score as a share of the winner's. The winner is always 100%.

Expected loss

How much performance you would give up on average by backing a variant that turns out not to be the best. The test stops when the winner's figure is small enough to act on.

Signal

The four-level rating of how conclusive a result is: Decisive, Strong, Indicative or Inconclusive.

Baseline

Your current message or creative, run as a variant so every challenger is measured against it.

Benchmark

A reference rate from outside the test: your own campaigns with the same audience, or typical rates for the category.

Segment

The audience a test runs with, defined by the targeting you already plan around. Each segment gets its own result.