Minimum Detectable Effect Explained
Minimum detectable effect, often shortened to MDE, is the smallest lift or drop you want your experiment to be able to detect. It is one of the strongest drivers of A/B test sample size.
What MDE really means
MDE is not your prediction of what will happen. It is the threshold at which a difference becomes important enough to matter for a decision.
That makes it a planning assumption about business relevance, not just statistics.
Why small MDEs are expensive
Detecting a tiny change requires a lot of data because small effects are harder to separate from noise. Teams often choose unrealistically small MDEs and then discover the traffic requirement is too high.
A more realistic MDE often makes a test much more feasible.
How to choose one
A practical MDE should reflect what would change a decision. If a 0.2 percentage-point lift would not justify rollout effort, there may be little value in designing the experiment around detecting that change.
Choose an effect size that is both meaningful and realistic for your traffic.
How teams choose a realistic MDE
A realistic MDE usually comes from combining business impact with traffic reality. If a change is too small to matter commercially, it may not be worth building a test around even if it is statistically interesting.
The opposite mistake is choosing an unrealistically large MDE just to make the test shorter. That can make the experiment blind to improvements that would actually be valuable.
- Tie the MDE to business impact, not curiosity alone
- Check that the target effect is plausible for the experiment
- Avoid shrinking the MDE until runtime becomes impossible
- Avoid inflating the MDE just to force a smaller sample