Read the conditional direction correctly
A p-value of .03 means that results at least as extreme as the observed result would occur with probability .03 under the null model and the test’s other assumptions. In shorthand:
P(data this extreme or more | null model), not P(null hypothesis | observed data).
Reversing those two probabilities is the most common interpretation error. The test conditions on the null; it does not assign posterior probability to it.
The phrase “as extreme or more extreme” depends on the test statistic and alternative. A two-sided test counts relevant extremes in both directions. A one-sided test counts the prespecified direction. That choice belongs to the analysis design and cannot be reconstructed from the p-value alone.
Alpha and p are different quantities
| Quantity | When it is set or observed | What it does |
|---|---|---|
| Alpha (α) | Chosen before interpreting the result | Supplies a decision rule for a defined testing procedure |
| P-value | Calculated from the data and model | Summarizes incompatibility between the data and null model |
If α = .05 and p = .03, the result crosses that prespecified rule. If p = .08, it does not. Neither statement tells you whether the effect is large or useful.
The .05 boundary is a historical convention. Fisher described .05 as a convenient point in significance testing, not a natural boundary between truth and falsehood.2 Modern ASA guidance recommends avoiding scientific conclusions based only on whether a p-value crosses a bright line.3
Why .049 and .051 are nearly the same evidence
P-values vary continuously. Results of .049 and .051 are much closer to one another than the labels “significant” and “not significant” suggest. Sampling variation, modeling choices, measurement decisions, and multiplicity can move a result across the line without creating a scientific discontinuity.
Report the estimate, uncertainty, exact p-value, design, and analysis choices. Confidence interval interpretation shows the compatible range that a threshold label hides. Statistical versus practical significance separates statistical evidence from importance.
What a small p-value does not establish
A small p-value does not by itself establish:
- a large effect;
- an important effect;
- a causal relationship;
- a well-specified model;
- absence of bias or selective reporting;
- replicability;
- a low probability that the null is true.
The ASA statement explicitly separates statistical significance from the size or importance of an effect and warns that a p-value alone is not a complete measure of evidence.1
What a non-significant result permits
A p-value above α means the selected procedure did not reject the null at that rule. It does not prove no effect, confirm equivalence, or show that the study had enough information to distinguish small from meaningful effects.
Read the confidence interval. A wide interval may include both zero and practically important values, which indicates imprecision. A narrow interval contained within prespecified equivalence bounds can support a different, affirmative conclusion—but that requires an equivalence design, not ordinary nonsignificance.
Avoid “marginally significant” as a way to rescue a near-threshold result. State the exact value and the continuous evidence instead.
Multiplicity changes the decision context
When many hypotheses are tested, the chance that at least one crosses an unadjusted threshold can rise. A lone p = .04 means something different when it is the only prespecified comparison than when it is selected from hundreds of exploratory tests. The correction, model, and family of claims need to be reported.
ANOVA table interpretation explains how an omnibus p-value and later comparisons answer different questions. effect-size interpretation covers magnitude conventions that p-values cannot provide.
A compact reading sequence
- Name the null model and test statistic.
- Check whether the test is one- or two-sided.
- Read the exact p-value, not only a star or category.
- Compare with a prespecified alpha only when a decision rule is relevant.
- Inspect the estimate and confidence interval.
- Account for multiplicity and analysis choices.
- Describe practical meaning separately.
A defensible reporting sentence
“The estimated difference was 2.4 units (95% CI [0.3, 4.5]), with p = .027 from the prespecified two-sided test.” Then state what the estimate means on its original scale. Do not translate p = .027 into a 97.3% probability that a hypothesis is true.