A/B Test Sample Size Examples
Examples make A/B test planning easier because they show how assumptions change the traffic requirement. This page focuses on intuition rather than formal proof.
Example: modest baseline, modest lift
Imagine a page with a 10% baseline conversion rate and a target lift of 2 percentage points. That kind of setup often produces a manageable but still meaningful sample requirement.
It is a good example because it sits in the range many growth and product teams actually work with.
Example: small baseline, tiny lift
Now imagine a 2% baseline and a 0.3 percentage-point lift. The required sample rises quickly because the effect is small and the baseline is relatively low.
This is why very small target effects can make experiments impractical for low-traffic experiences.
The practical lesson
Examples like these show that the key question is not just how much traffic you have. It is whether the effect you care about is large enough to detect within the traffic you can realistically collect.
That is what makes sample size planning a strategic filter for experimentation.
How to learn from examples without copying them
Examples are most useful for building intuition about how assumptions interact. They show how baseline, effect size, and power can move the traffic requirement much more than people expect.
What examples should not do is replace your own planning inputs. Even a very similar-looking test can need a different sample if the audience, metric, or baseline behaves differently.
- Use examples to understand direction, not to copy a sample target
- Compare your own baseline and MDE against the example setup
- Treat example numbers as illustrative rather than prescriptive
- Run your own scenario in the calculator before launching