Sample ratio mismatch (SRM)

When traffic actually observed across cohorts deviates from the intended split — a red flag for a broken randomizer, not a real effect.

Sample ratio mismatch is what you get when the traffic actually observed across cohorts doesn't match the intended allocation weights — a campaign configured for a 50/50 split that's actually landing 55/45 or worse. It's detected with a simple chi-squared goodness-of-fit test against the configured weights, and it's one of the few checks worth running before you trust any other number the experiment reports.

SRM almost never means the effect is real; it means something upstream of the analysis is broken — a redirect that fails differently for one cohort, a caching layer serving stale assignments, a client-side randomizer that isn't actually random. Finding SRM should stop you from reading the results at all until the cause is found, not become a footnote in the readout.