Nexframe
Wilson intervals
A 95% range for every creative's rate, honest even with few clicks. Rows whose range touches the average are faded.
Roblox Ads CTR test calculator
Which adreally won?
Paste your Roblox Ads Manager results. Get a verdict with real statistics, and how many impressions you still need. Free.
Try it with real data from our own campaigns:
Runs in your browser. Nothing is uploaded, and no account is needed.
Our horror campaignReal counts, 237,041 impressions
+34%more players per impression from T4, the creative that lost on clicks to T1.
White line: campaign average (1.80%). Faded rows can't be told apart from it yet. Load this campaign in the calculator to see every step.
Our own Roblox Ads
The creative that won on clicks lost on players.
Real numbers from two of our Roblox Ads campaigns. Load them in the calculator above to see every step.
+13%T1's lead over T4 in CTR. On plays per impression, T4 was 34% ahead.
84%of simulations where 16 identical creatives still produced a 'significant' best-vs-worst gap.
24%fake winners when you peek every 1,000 impressions and stop at the first 'win' (4.5% if you look once).
30xCPM gap between two of our own campaigns. A sample size is only priced right at your own CPM.
Step 3: Test it for real on Roblox
Run a clean test
The Ads Manager is not an A/B test: it shifts impressions toward the creatives it likes. Five rules give you numbers worth reading.
- 1Make one campaign per creative: same objective, same targeting, same daily budget. With one image per campaign, the Ads Manager cannot shift traffic between your creatives.
- 2Turn them all on in the same minute, and never edit them while the test runs. An edit changes the audience and the auction, so it starts a new test.
- 3Budget at least 10 Ad Credit per campaign (the Ads Manager minimum), and plan the size with the planner on this page before you spend.
- 4Wait for about 30 plays per creative before you look. The final size comes from the planner, not from a rule of thumb.
- 5Paste the creatives table here and judge on plays per impression. CTR alone can pick the creative that brings fewer players.
Step 4: Read the result
Standard A/B statistics, made for the Ads Manager table
No black box: the same formulas a data team would use, running in your browser. It judges on plays per impression when your table has plays, because that is the number that pays, and keeps CTR and plays per click as diagnostics. Every formula is written out on the methodology page.
Holm correction
Comparing the leader with 15 creatives is 15 chances to get lucky. Holm raises the bar to match, so 0.042 becomes 0.64.
P(best) and expected loss
Bayesian draws from each creative's Beta posterior, with a fixed seed: the same numbers always give the same answer.
How much is real
Empirical Bayes splits the spread between creatives into real difference and noise, and pulls small samples toward the mean.
Sample planner
Impressions per creative for the difference you care about, priced at your own CPM, and the reverse: what your volume can detect.
Always-valid p-values
Peeking every day is safe with an mSPRT p-value. A normal p-value checked daily crowns fake winners.
WinnerSurvives Holm, the lift interval stays above zero, enough plays, and the planned sample (or an always-valid p) is reached.
Clicks vs playsThe CTR leader brings significantly fewer players per impression.
Practical tieThe whole lift interval sits within 10%: test a new idea instead.
Likely, not provenAhead with P(best) of 80% or more, but some condition is missing.
Too earlyEverything else, with the smallest difference your volume can detect.
Questions
Common questions
How do I know which Roblox ad creative actually won?
Paste the creatives table from the Roblox Ads Manager. The calculator compares every creative with the leader using a two-proportion z test with Holm correction, puts a 95% interval on the lift, and only says Winner when the difference survives that correction, the interval stays above zero, each creative has at least 30 plays (or clicks) and the planned sample was reached. Otherwise it says Likely, Practical tie or Too early - and how many more impressions you would need.
Why judge on plays per impression instead of CTR?
Because a click that does not turn into a play is a promise the game did not keep. In our own horror campaign, the creative with the best CTR beat another one on clicks by 13%, yet brought 25% fewer players per impression. When your table has plays, the calculator decides on plays per impression and shows CTR and plays per click as diagnostics.
How many impressions does a Roblox ad test need?
It depends on your base rate and on the smallest difference you care about. With a CTR of 1.74%, seeing a 20% difference between two creatives takes about 24,000 impressions each; with 4.5%, about 9,000. With more creatives the bar goes up, because every extra comparison is another chance to find a fake winner. The planner on this page does the math with your own numbers and your own CPM, which can differ 30 times between two campaigns of the same developer.
Is my data uploaded?
No. The calculation runs in your browser and nothing is sent to our server. The share link carries the numbers after the # in the address, a part that browsers do not send to servers. Thumbnails you attach stay on your device.
Why did my creatives get such different numbers of impressions?
Roblox says the Ads Manager starts by serving creatives evenly and then shows the most engaging ones more to each group of players. So creatives with very different volumes may have reached different audiences, and part of the difference can be audience, not image. The calculator flags this with a sample ratio check. For a clean test, run one campaign per creative.
What does 'always-valid p-value' mean?
If you check a test every day and stop the moment it looks significant, you will crown a fake winner far more often than 5% of the time - in our simulation, 23.9% instead of 4.5%. An always-valid p-value (mSPRT) is built to be checked as often as you like. The calculator calls a winner early only if that p-value, after Holm correction, is below 0.05.
Can it tell me how many clicks my next ad will get?
No. Every number here describes what already happened, with its uncertainty. A lift of +20% (95% CI +5% to +37%) means the true difference during your test was probably in that range - not that a new campaign will repeat it.
Test a new idea, not a new arrangement
Check it at Home size with the free Roblox thumbnail tester, or make a genuinely different one with the Roblox thumbnail maker.