P-Value
01 Definition
P-Value meaning: The probability of seeing a result this extreme if the change truly had no effect.
A p-value measures how surprising your data would be if the null hypothesis, meaning no real difference, were true. A small p-value suggests the result is unlikely to be random chance alone. Common practice treats a p-value below 0.05 as statistically significant, but this cutoff is a convention, not proof of importance.
Why it matters
How P-Value fits the work
The p-value gives a common yardstick for deciding whether a test result is strong enough to act on rather than likely random variation.
In context
The test returned a p-value of 0.03, below the 0.05 threshold, so the team treated the lift as statistically significant.
Practical note
A p-value does not tell you the size of an effect or the chance your hypothesis is true. A significant result can still be too small to matter for the business.