How to use it
- Pick the layout and paste. It is the same format the error bar chart maker takes.
- Say whether the data are paired — the same subject measured before and after, for instance. Equal group sizes are not evidence of it.
- Pick the control group. Every other group is compared against it.
- Read “why this test” first. It matters more than the result.
- Look at the recommended answer and the other one together, and think again about the assumptions if they differ.
The formulas, and what to watch for
The value of this page is not the arithmetic. It is deciding what arithmetic to do.
What the choice rests on
Not a normality p value — at bench sample sizes that number is not evidence. Only grounds that can be defended.
- Fewer than 8 per group
- normality cannot be judged → rank test
- Levene rejects
- a test that does not assume equal variance (Welch)
- Otherwise
- the classic parametric test
Equal variance — Levene, median-centred
An ANOVA on the absolute deviations from each group's median. The median rather than the mean, because centring on the mean makes the test itself sensitive to the non-normality it is meant to be checked alongside.
- Statistic
- F on |xᵢⱼ − medianⱼ|
Rank tests
With no ties and a small sample the permutations are counted and the p value is exact. Otherwise a normal approximation with continuity and tie corrections.
- Mann-Whitney
- c(m,n,u) = c(m−1,n,u−n) + c(m,n−1,u)
- Smallest p at 3 vs 3
- 2 / C(6,3) = 0.1
What to watch for
- ⛔ There is no Shapiro-Wilk. Its coefficients could not be sourced and were not copied out of somebody's code. What is shown instead is stated on the page.
- With few replicates normality *cannot be tested*. “p > 0.05” then means “we could not tell”, not “it is normal”, and this tool says which.
- Pairing is a question about the design. The numbers cannot answer it, and running a paired test on unpaired data throws away degrees of freedom.
- When the recommended and alternative answers differ, do not pick the smaller p. That they differ is itself the finding: the result is leaning on the assumption.
- The correction on the control comparisons is Holm, not Dunnett's own critical values, and the page says so.
- At three against three a rank test cannot reach significance at all. That is a limit of the design, not of the data.
Frequently asked questions
Its coefficients could not be obtained from a primary document — the papers are paywalled and the university copy that surfaced serves an expired certificate. Copying the numbers out of somebody else's implementation is not something this site does, so skewness, kurtosis and the Q-Q correlation are shown instead and the recommendation rests on sample size and equal variance.
It can be. But that p value stands on an assumption of normality that three observations cannot confirm. The rank-based answer is shown beside it — and at three against three that one cannot go below 0.1.
It depends on how you judge the assumptions, and the tool does not judge for you. Only do not pick the smaller p. Reporting that they differed is the honest thing.
Dunnett's design — every group against one control — yes. The correction is Holm rather than Dunnett's own critical values, which need multivariate t tables that could not be sourced. Holm is valid and a little conservative.