Google Ads Experiments: Testing Changes on a Split of Traffic

Everything below comes from Google Ads Help: About the Experiments page, About custom experiments, Set up a custom experiment, Set up an ad variation, Create a video experiment, Monitor your experiments, Apply a custom experiment and Google's experiment best practices. The general method of testing (one change, a fixed sample, no early stopping) is in our A/B testing guide; this page sticks to how Google Ads does it.

What a Google Ads experiment is

Google describes it this way: The experiment shares your original campaign’s traffic (and budget) and lets you test changes to your campaigns so that you can make more informed decisions on which tactics give you a better return on investment. You keep the original campaign (the control arm) and Google runs a trial version with your change (the treatment arm) at the same time, on the same auctions, so seasonality and competitor moves hit both sides alike.

That is the advantage over the usual alternative, changing a setting and comparing this month with last month. A before-and-after comparison mixes your change with everything else that moved in between. An experiment separates them. Everything lives under Experiments in the Campaigns menu, which Google formerly called drafts and experiments.

The experiment types

The Experiments page groups tests by what they change:

• Custom experiments: Google says these are typically used to test Smart Bidding, keyword match types, landing pages, audiences and ad groups. You pick a base campaign, change the setting you want to test, and Google creates the trial campaign.
• Ad variations: for Search campaigns. They test a change to ad text or URLs across many ads at once: find and replace a phrase (Google's example swaps "Book now" for "Call now"), add, remove or pin headlines and descriptions in responsive search ads, or change final URLs and display paths. Variations are case sensitive.
• Video experiments: compare video ads on YouTube. The basic A/B test video assets option duplicates one Video Reach or Video View campaign and changes only the creative. The Custom option tests any variable with up to 10 arms. Success metrics include CTR, conversion rate, cost per conversion, CPV, CPM and Brand Lift, though Google notes Brand Lift is not available to every account.
• AI Max experiments: a one-click test for Search campaigns of features such as search term matching and asset optimization. Instead of copying the campaign, it splits the current campaign 50/50.
• Performance Max experiments: measure the uplift of adding Performance Max to your campaign mix, or of switching from Standard Shopping to Performance Max.
• Demand Gen and App asset experiments: test which images or videos work in Demand Gen campaigns, and which text and images drive installs in App campaigns.

Google's own advice on choosing between the first two: to test Search ad creative, use ad variations; to test specific ad groups rather than the whole campaign, use a custom experiment.

Which campaigns can run a custom experiment

Google's pages list Search, Display, Demand Gen and Video campaigns (the set-up page also names Hotel Ads). Custom experiments are not available for App or Shopping campaigns; App campaigns use App asset experiments instead. For Video, only auction-based campaigns qualify, not reservation products such as YouTube Select or Masthead.

The base campaign must be active, use its own daily budget rather than a shared budget, and use a standard bid strategy rather than a portfolio strategy. Google also notes that the base campaign cannot contain the retired Text ads or Expanded Text ads, including in paused and removed ad groups, and that the system can take 24 to 48 hours to update after you remove them. If the base campaign is paused or ends before the experiment does, the experiment stops serving.

A few reports and features do not work inside custom experiments, including auction insights, search terms and category reports, keyword diagnosis, ad schedule reports, scheduled email reports, shared budgets and ad customizers that target a specific campaign or ad group. You can schedule up to 5 experiments, each covering several campaigns (up to 100), but Google says only one experiment runs at a time.

Traffic split: cookie-based or search-based

When you set up the test you choose what share of the original campaign's traffic and budget the experiment gets. For Search, you also choose how people are assigned:

• Cookie-based: each user sees only the original or only the experiment, however many times they search. Display campaigns always use this. Google says that with audience lists you should have at least 10,000 users in the list for a cookie-based custom experiment, or results may be less accurate.
• Search-based: each search is assigned at random, so the same person can see both versions if they search more than once. Keep that in mind when people usually search several times before they buy.

Ad variations always split by cookie, which Google says means users may see only one version of the ad. A split sets auction eligibility, not equal results. Google is explicit that a 50/50 split doesn't guarantee equal impressions or spend: each arm wins or loses auctions on its own Ad Rank, a side that reaches its daily budget first stops for the day, and automated bidding shifts volume while it learns. Google suggests allowing 7 to 14 days for the treatment arm to stabilize, and checking whether its bid strategy shows Learning before reading anything into early numbers.

Setting up a custom experiment

The quickest route is from the campaign itself: change a setting and choose Save as experiment instead of saving it. From the Experiments page the steps are:

1. Select the plus button in the All experiments table and choose Custom.
2. Choose the experiment and the variable to test.
3. Select the original campaigns. Only eligible ones appear; for some tests the list is limited to campaigns that use Smart Bidding and have exact or phrase match keywords.
4. Pick a success metric. Google lets you name up to two primary metrics and tailors the report to them.
5. Decide on auto-apply: whether the change should be applied to the original campaign automatically if the treatment wins, and whether that needs statistical significance or only a directional result.
6. Set the start date, duration and a name, then select Schedule.

New ads in the experiment go through review, which can take a while on large campaigns, so Google suggests scheduling the start a little in the future. If creation fails, the usual causes Google lists are a removed or shared-budget base campaign, another experiment already scheduled for those dates, unsupported features, or keywords with policy issues that were not copied across.

Reading the results

Open the experiment from the Experiments page to see a scorecard and a summary table. For each metric the scorecard shows the experiment's own figure, then the estimated difference against the original and a confidence interval. Google's example: +10% for clicks with an interval of [+8%, +12%] means the experiment probably got between 8% and 12% more. The default confidence level is 80%, which you can change, and a blue asterisk marks a statistically significant result. A ‑‑ means there is not enough data yet.

Conversion metrics such as conversions, conversion rate and cost per conversion only appear if conversion tracking is set up, which is a good reason to fix tracking before running bid tests. The Results column names the control or the treatment as the winner, or says no clear winner. Google recommends letting an experiment run 2 to 3 weeks; for undecided results it suggests a bigger budget or more time, and for video experiments that are still undecided it suggests at least 4 to 6 weeks.

When a result is not significant, Google lists the likely reasons: the test has not run long enough, the campaign has too little traffic, the split was too small, or the change simply made no real difference. Video experiments label results by confidence: from 70% they are directional, and at 95% Google calls them conclusive. Broad match experiments add incremental queries: up to 5 converting search terms that matched the broad keyword and no other keyword in the account during the test.

Apply, convert or end

When the experiment is done you have three choices:

• Apply to the original campaign: the experiment ends and the tested changes are written into the original campaign, which keeps that campaign's history. Google suggests this when you want to keep what was learned during a learning period.
• Convert to a new campaign: the original campaign is paused and the experiment carries on as a normal campaign with the same budget and dates. Google suggests this when you tested a new campaign structure. Check the new campaign afterwards to make sure every change came across.
• End: change the end date or choose End now, and nothing is applied.

In every case the performance data of both arms is kept. Auto-apply, where available, is on by default: if the treatment's results are favorable, Google applies it and moves 100% of the traffic to it. You can turn it off from the report page at any time. It works much like auto-applied recommendations, so decide on purpose whether you want it. Video experiments are different: you end them by hand, and you choose whether to copy the winning assets into the control campaign.

Habits that keep results readable

Google's best-practice page boils down to four habits:

• Write the hypothesis first, tied to a business goal. Google's example: switching to Target ROAS will bring higher conversion value than Target CPA.
• Change one variable. If you change both the headline and the bid strategy you cannot tell which one made the difference.
• Pick the deciding metric before you start, usually your main goal such as total sales or cost per action, and let it name the winner.
• Leave the base campaign alone while the test runs. Google says changes to the original campaign are not copied to the experiment unless sync is on, and changes on either side make results harder to read.

Keep a short log of each test: what changed, dates, the result and what you did with it. It saves repeating tests and shows which ideas tend to work in your account. For small accounts the honest limit is traffic: a campaign with a handful of conversions a month may never reach significance, and a longer test or a broader change is then more useful than a fine-grained one.

Frequently asked questions

What are Google Ads experiments?

A way to test a change, such as a new bid strategy, landing page or ad text, on a share of a campaign's traffic and budget, side by side with the original campaign, and then apply the change only if it performs better.

How long should a Google Ads experiment run?

Google recommends 2 to 3 weeks for most experiments and allowing 7 to 14 days for the treatment arm to stabilize. If there is still no clear winner, it suggests more budget or more time; for video experiments, at least 4 to 6 weeks.

Can I run an experiment on a Shopping or App campaign?

Not as a custom experiment. Google says custom experiments are not supported for App or Shopping campaigns. App campaigns have App asset experiments, and Performance Max experiments can compare Standard Shopping with Performance Max.

Does a 50/50 split mean each side gets half the impressions?

No. Google says the split controls auction eligibility but does not guarantee equal impressions or spend, because each arm competes in auctions on its own bids, ad quality and budget.

What happens when I apply an experiment?

The changes are copied into the original campaign and the experiment ends. Alternatively you can convert it into a new campaign, which pauses the original. Performance data for both sides is kept either way.

An experiment is only as good as the page people land on and the conversions you track. See our website development service and all our services, or email [email protected].