Free Tool

A/B test significance calculator

Is your new landing page, headline or ad really better, or did it just get lucky? Enter your results to find out. Planning a test instead? Work out how much traffic you'll need first.

1 · Check your results

Confidence

A · Control Visitors Conversions

Conversion rate —

B · Variant Visitors Conversions

Conversion rate —

—

Uplift (B vs A)

—

Confidence: —
p-value: —

2 · Plan a test: sample size

Current conversion rate (%)

Smallest uplift worth detecting (%)

Relative, e.g. 20% means 3% → 3.6%.

Statistical power

Visitors per day (total)

Split evenly across 2 variants.

Visitors needed per variant

—

Total visitors: —
Estimated duration: —

How to read your A/B test

An A/B test splits traffic between two versions of a page, ad or email and compares how often each converts. The hard part isn't running the test; it's knowing when a difference is real.

Uplift

The relative change in conversion rate from A to B. Going from 3.0% to 3.8% is a 26.7% uplift. The range underneath shows where the true uplift probably lies: if it crosses zero, B might actually be worse.

Confidence and p-value

The p-value is the probability of seeing a difference this large if the two versions actually performed the same. Confidence is 1 − p. If confidence is above the level you chose (usually 95%), the result is statistically significant.

Plan before you test

Use the sample size section before launching. Decide the smallest uplift that would be worth acting on, see how many visitors that takes, and commit to running until you reach it. Stopping early when the result "looks good" is the most common way tests produce false winners.

Practical tips

Run tests for at least one full week, even with lots of traffic, so weekday and weekend visitors are both included.
Test one big idea at a time, such as a new headline angle or a shorter form, rather than tiny tweaks. Big changes produce big, detectable differences.
Low traffic? Test higher up the funnel (ad clicks, form starts) where there are more events, or test bolder changes.

FAQ

A/B testing questions

Can't find what you need? Ask us directly.

What does statistically significant mean?

It means the difference between your variants is unlikely to be down to random chance. At 95% confidence, you'd expect a difference this large to appear by luck less than 5% of the time if the two versions actually performed the same.

How many visitors do I need for an A/B test?

It depends on your current conversion rate and the smallest improvement you care about. Small conversion rates and small improvements need a lot of traffic. Use the sample size section to get a number before you start, rather than stopping when the result looks good.

Can I stop a test as soon as it's significant?

It's risky. Checking results repeatedly and stopping at the first significant reading ("peeking") makes false winners much more likely. Decide your sample size up front, run the test for at least one full week to cover weekday and weekend behaviour, then check.

What is a good confidence level?

95% is the usual standard. 90% is reasonable for low-risk changes where you want faster decisions; 99% is worth it for changes that are expensive to roll out or reverse.

Which statistical test does this use?

A two-tailed two-proportion z-test with pooled variance, the standard approach for comparing two conversion rates. The sample size calculation uses the matching formula for the confidence and statistical power you choose.