Kiwistat Statistics Guide

This guide explains the statistics behind Kiwistat: what each test does, when to use it, its assumptions, and how to interpret and report the output. It assumes a strong science background but takes nothing for granted statistically. For the application itself — entering and importing data, tabs, graphs, saving — see the User Guide.

1A short statistics primer

Samples estimate populations

Your measurements are a sample drawn from a larger population (all possible measurements at that site, of that species…). Statistics asks: what can this sample tell us about the population, given that another sample would have come out a bit different?

Describing variation: SD vs SE

• Standard deviation (SD) describes how spread out the individual measurements are.
• Standard error (SE) = SD ⁄ √n describes how precisely you know the mean. It shrinks as you take more measurements; SD does not.

Error bars on Kiwistat's bar charts show mean ± SE by default. Say which you plot in your figure caption — reviewers check.

What a p-value actually is

Common misreadings to avoid:

• p is not the probability that H₀ is true, and 1−p is not the probability your hypothesis is right.
• p = 0.06 vs p = 0.04 is not “no effect” vs “effect” — p-values are continuous evidence. Report the exact value.
• Statistical significance ≠ practical importance. With large n, tiny irrelevant differences become “significant”; with small n, large real effects can be missed (low power). Always look at effect sizes and means, not just p.
• Running many tests inflates false positives: at α = 0.05, about 1 in 20 truly-null comparisons will come out “significant” by chance. This is why post-hoc tests adjust for multiple comparisons.

Assumptions matter

The parametric tests in Kiwistat (ANOVA, Pearson correlation, curve fitting) assume, to varying degrees:

1. Independence — each observation is a separate experimental unit. (Ten readings of the same plant are not 10 independent replicates — that's pseudoreplication, and no software can fix it.)
2. Normality — the residuals (scatter around group means) are roughly bell-shaped. Check with Normality & Transformation. ANOVA is fairly robust to moderate non-normality, especially with balanced group sizes.
3. Equal variances — the scatter is similar in every group. Kiwistat checks this automatically with Levene's test and tells you what to do if it fails.

Why log-transform?

Environmental measurements are often right-skewed: many small values, a few large ones (concentrations, counts, biomass, rainfall). Log-transforming such data usually makes it more symmetrical and evens out variances — fixing two assumptions at once. Use Use log₁₀ values for statistics in the Column pane; back-transformed results are geometric means.

2Choosing the right test

3Normality & transformation

Runs the Shapiro–Wilk test on your variable — the most powerful general normality test for small-to-medium samples — plus a candidate transformation (log₁₀, ln or √), and recommends whether transforming helps.

• W close to 1 = consistent with normal; p < 0.05 = significant departure from normality.
• Skewness: 0 = symmetrical; positive = long right tail (log usually helps); negative = long left tail.
• If the data will feed an ANOVA, set Group by to your factor — the assumption is normality within groups (of the residuals), not of the pooled data. A mixture of groups with different means can look non-normal even when every group is perfectly normal.

The graph tab shows a normal Q-Q plot: sample values against theoretical normal quantiles. Points along the dashed line = normal; a bow shape = skew; S-shape = heavy or light tails.

4One-way ANOVA

Analysis of variance tests whether the mean of a numeric response differs among the levels of one factor. It works by comparing variation between group means with variation within groups: if groups differ more than their internal scatter can explain, the ratio F is large and p is small.

Setup: Response = your measurement column; Factor = your grouping column; choose a post-hoc test (below) and α. Each group needs at least 2 observations, and the response must be numeric.

The ANOVA table:

A significant ANOVA says “at least one group differs” — it does not say which. That is the job of the post-hoc test.

5Post-hoc tests: which one?

Post-hoc tests compare every pair of groups while controlling the family-wise error rate (the chance of any false positive across all comparisons). They differ in how strictly they control it:

Letter groupings

Results and bar charts carry compact letter displays: groups sharing a letter are NOT significantly different. So “a, ab, b” means the outer groups differ, and the middle group can't be distinguished from either. Letters start at “a” for the highest mean.

6Levene's test & unequal variances

With every ANOVA, Kiwistat automatically runs Levene's test (Brown–Forsythe, median-centred — the robust version) on the equal-variance assumption. It is itself an ANOVA on the absolute deviations from each group's median.

• p ≥ 0.05 — variances are homogeneous; carry on.
• p < 0.05 — variances differ. The standard F-test can then be misleading, so Kiwistat also reports Welch's ANOVA, which does not assume equal variances — quote Welch's F and its (fractional) degrees of freedom instead — and recommends the Games-Howell post-hoc. If the spread grows with the mean (very common), ticking Use log₁₀ values for statistics on the response column often fixes the problem at the source; re-run and check Levene again.

7Two-way ANOVA

Tests two factors at once and — the real payoff — their interaction:

• Main effect A: averaged over B, do A's levels differ?
• Main effect B: averaged over A, do B's levels differ?
• A × B interaction: does the effect of one factor depend on the level of the other? (e.g. fertiliser boosts growth in species 1 but not species 2).

Requirements: replication inside every factor combination, ideally balanced (equal n per cell — Kiwistat's sums-of-squares are exact for balanced designs). Levene's test here checks variances across all combinations. The post-hoc applies to Factor A's main effect.

8Blocked ANOVA (Randomised Complete Block Design)

Field and glasshouse trials rarely have uniform conditions: soil, light, or time-of-day varies across the experiment. The randomised complete block design groups experimental units into blocks that are internally similar (a field strip, a bench, a sampling day), and applies every treatment once within each block. The analysis then removes the block-to-block variation from the error term, making the treatment comparison much more sensitive.

Setup: a numeric response, a treatment factor, and a block factor — each treatment should appear once in each block. The results table has three rows:

• Treatment — the effect you care about; report this F and p.
• Block — a significant block effect confirms blocking was worthwhile (it soaked up real variation).
• Error — the residual scatter, now free of block differences.

Post-hoc comparisons and letter groupings apply to the treatment means, exactly as in one-way ANOVA. The example dataset “Wheat yield trial (RCBD)” lets you compare a blocked analysis against a naïve one-way ANOVA on the same data — the blocked test has a smaller error and a sharper treatment result.

9Repeated measures ANOVA

When you measure the same subject more than once — a plant every fortnight, a plot each season, a patient before and after treatment — the measurements are not independent. Two readings from the same plant are more alike than two readings from different plants, and an ordinary ANOVA, which assumes every observation is independent, gets the error term badly wrong. A repeated measures ANOVA removes each subject's own level from the error, exactly as blocking does — in fact a simple repeated measures ANOVA is arithmetically an RCBD with subjects as the blocks.

How to lay the data out

Kiwistat expects wide format: one row per subject, and one column per occasion. This is how time-series data usually leaves a spreadsheet or a logger.

Setup: tick the measurement columns in time order; optionally name a subject/plot ID column (used only for labelling) and a between-subjects factor such as treatment. Rows missing any occasion are dropped whole — the design needs a complete set per subject — and the count of dropped rows is reported.

Reading the table

• Time (occasion) — does the measurement change over the series? This is the within-subjects effect, tested against the time × subjects error.
• Subjects — the variation between individuals that has been taken out of the error. It is not usually a hypothesis of interest, but a large value shows why pairing mattered.
• With a between-subjects factor the table splits in two. Between subjects tests the treatment against subject-to-subject variation; within subjects tests time and the Time × treatment interaction. That interaction is usually the real question: do the groups follow different trajectories?

Sphericity — the assumption that catches people out

Repeated measures ANOVA assumes sphericity: every pair of occasions has the same variance of differences. Time series routinely break it, because measurements close together are more alike than measurements far apart. When sphericity fails, the F-test is too liberal — p-values come out smaller than they should be.

• Mauchly's test checks it. A significant result (p < 0.05) means sphericity is violated.
• Greenhouse–Geisser and Huynh–Feldt fix it by multiplying the degrees of freedom by an epsilon (ε ≤ 1). Kiwistat reports both, with fractional df — that is expected, not a bug.
• Convention: use Greenhouse–Geisser when ε < 0.75 and Huynh–Feldt when it is larger. Many authors report Greenhouse–Geisser regardless, because Mauchly's test has little power at small n.
• With only two occasions there is a single difference score, so sphericity cannot be violated and no correction is needed (that case is equivalent to a paired t-test).

The graph is a profile plot: mean ± SE at each occasion, joined, with one line per between-subjects group. Parallel lines mean no interaction; converging or crossing lines are the interaction made visible. The example dataset “Seedling height over time” is set up for this analysis.

10Non-parametric tests

When data are skewed, ordinal, riddled with outliers, or simply won't transform to normality — common with counts and concentrations — switch to a rank-based test. These convert values to ranks and ask whether one group tends to have higher ranks than another. They make no normality assumption and resist outliers.

Kruskal–Wallis (3 or more groups)

The non-parametric counterpart of one-way ANOVA. It reports a tie-corrected H statistic (compared to a chi-square distribution) and a p-value. A significant result means at least one group's distribution differs. Dunn's test then compares each pair with a Bonferroni adjustment, giving the familiar letter groupings on a boxplot. Report the medians (not means) as your summary.

Mann–Whitney U (exactly 2 groups)

The non-parametric counterpart of a two-sample t-test. It asks whether values in one group are systematically larger than in the other. Kiwistat reports U, a z-approximation, and a two-sided p-value with tie and continuity corrections (reliable for roughly n ≥ 8 per group).

11Linear regression

Where correlation measures how tightly two variables move together, regression fits the actual line y = a + bx and quantifies it: the slope b is the change in y per unit x. Kiwistat reports the intercept and slope with standard errors, t-tests and p-values, R², and a 95% confidence interval for the slope. The key test is whether the slope differs from zero (p for b).

Confidence vs prediction bands

On the graph tab you can show two shaded bands around the line:

• Confidence band — where the true regression line probably lies (narrow).
• Prediction band — where a new individual observation will probably fall (wide, because it also includes scatter around the line).

Diagnostic plots — always look at these

R² tells you how well the line fits, but not whether a line is appropriate. Two diagnostic panels appear beneath the fit:

• Residuals vs fitted — should be a shapeless horizontal band around zero. A U or hump shape means the relationship is curved (try curve fitting or a transform); a widening fan means variance grows with x (try log-transforming y).
• Residuals vs leverage — points plotted in red have a Cook's distance greater than 4/n, meaning they individually pull the line towards themselves. Check them for data-entry errors or genuine influential observations before trusting the fit.

12Curve fitting

Fits a model of Y against X by least squares and reports the equation, R² (fraction of the variation in Y explained) and adjusted R² (penalised for extra parameters — use this to compare models).

Auto picks the best adjusted R², but prefer a model with a mechanistic justification over a marginally better empirical fit — and never extrapolate beyond your data range.

13Correlation matrix

Pairwise correlation coefficients: r = +1 (perfect positive), 0 (no association), −1 (perfect negative). Bold cells are significant (p < 0.05); each pair uses all rows where both values are present. Choose the method with the Method selector:

• Pearson (default) measures linear association — a strong curved relationship can still give r ≈ 0, and one outlier can create or destroy it. Best for roughly normal data. Plot first.
• Spearman correlates the ranks instead of the values. It captures any monotonic relationship (consistently increasing or decreasing, even if curved), makes no normality assumption, and shrugs off outliers — a safer default for skewed environmental data.

• Correlation is not causation — both variables may follow a third (in environmental data, often temperature, depth or season).
• With many variables, some cells will be “significant” by chance (≈1 in 20 at α = 0.05).

14Principal Components Analysis

PCA condenses many correlated variables into a few new axes (principal components) that capture as much of the variation as possible. Kiwistat standardises each variable first (correlation-matrix PCA), so variables with big units don't dominate.

• Eigenvalues / % variance: how much variation each PC explains. An eigenvalue > 1 means the PC explains more than one original variable's worth. If PC1+PC2 explain, say, 70%+, the biplot is a faithful summary.
• Loadings (eigenvectors): how strongly each original variable contributes to each PC — use them to name the axes (“PC1 = overall nutrient enrichment”).
• Scores plot: each sample plotted on PC1–PC2; samples that plot together have similar overall profiles. Choose a grouping column to draw 95% ellipses per group.
• Rows missing any selected variable are dropped. The sign of an axis is arbitrary — “left” vs “right” has no meaning by itself.

15Power analysis & sample size

The best time to think about statistics is before you collect data. Power is the probability that your study will detect an effect of a given size if it is really there. An underpowered study wastes effort: a real effect goes undetected, and a non-significant result becomes uninterpretable (“no effect” or “not enough data”?). The convention is to design for 80% power.

Kiwistat handles two designs — two groups (t-test) and one-way ANOVA with k groups. Provide:

• The effect size you need to detect, either directly (Cohen's d for two groups, f for ANOVA) or as your expected group means and within-group SD — take these from a pilot study, the literature, or the smallest difference that would matter biologically.
• α (usually 0.05) and your target power (usually 0.8).
• Optionally a planned n per group, to read off the power you'd achieve.

The output gives the required n per group, the power of your planned n, and a power curve showing how power rises with sample size. Guideline effect sizes: d 0.2 small / 0.5 medium / 0.8 large; f 0.1 / 0.25 / 0.4.

16Exploring data: boxplots & scatterplot matrix

Two graph types under Test Type produce figures with no hypothesis test — for looking before you leap. Always explore your data this way first: it reveals skew, outliers, unequal spread and non-linear relationships that decide which formal test is appropriate.

Boxplots

For one numeric variable, optionally split by a factor. The box spans the interquartile range (middle 50% of the data), the heavy line is the median, the whiskers reach the most extreme values within 1.5 × IQR of the quartiles, and points beyond are drawn as outliers. Tick Show raw data points in the graph options to overlay every observation (jittered) — increasingly expected by journals. A median sitting off-centre in its box signals skew; boxes of very different heights signal unequal variances.

Scatterplot matrix

For several numeric variables, this grid plots every pair against each other at once. Scan it for straight-line trends (candidates for regression or correlation), curves (curve fitting), clusters, and outliers — the quickest way to get to know a multivariate dataset.

17Reading & reporting results

• Red p-values are significant at 0.05. Report exact values (“p = 0.003”), reserving “p < 0.001” for very small ones.
• Always report the test, the statistic with its degrees of freedom, the p-value, and n — e.g. “F_2,15 = 13.3, p < 0.001, n = 6 per site”.
• Mention the assumption checks: “variances were homogeneous (Levene's test, p = 0.32)” or “variances were unequal, so Welch's ANOVA and Games-Howell comparisons were used”.
• If you used log statistics, say so: “data were log₁₀-transformed for analysis; means shown are geometric”.
• The Sig. figs box in the app's header controls display rounding everywhere; underlying values keep full precision.

For a ready-made methods paragraph describing Kiwistat's statistical implementation, see Describing Kiwistat in publications in the User Guide.

18Glossary

α (alpha)

The significance threshold, usually 0.05: the false-positive rate you accept.

ANOVA

Analysis of variance — compares means of 2+ groups via a variance ratio (F).

Degrees of freedom (df)

The number of independent pieces of information behind a statistic; quoted with F and t.

F statistic

Ratio of between-group to within-group variance; large F → group means differ.

Factor

A categorical explanatory variable (site, species, treatment). Its values are levels.

Block

A group of experimental units treated as internally uniform (field strip, day, bench); blocking removes their shared variation from the error.

Censored value

A measurement known only to be below (or above) a limit, e.g. “<0.05”; substituted for analysis per a stated rule.

Cook's distance

How much a single point influences a regression fit; values > 4/n flag influential points.

Effect size

The magnitude of a difference or relationship, independent of sample size (e.g. Cohen's d, f, r).

Family-wise error rate

Chance of at least one false positive across a set of comparisons; post-hoc tests control it.

Geometric mean

Back-transformed mean of logs; the natural “average” for log-scale data.

Interaction

When one factor's effect depends on another factor's level.

IQR (interquartile range)

Q3 − Q1, the spread of the middle 50% of the data; the height of a box in a boxplot.

Kruskal–Wallis test

Rank-based (non-parametric) alternative to one-way ANOVA; reports H.

Leverage

How far a point's x-value is from the mean; high-leverage points can dominate a regression.

Levene's test

Tests equality of variances between groups (Kiwistat uses the robust Brown–Forsythe form).

Mann–Whitney U

Rank-based (non-parametric) alternative to a two-sample t-test.

Non-parametric test

A test that works on ranks and makes no distributional assumption (Kruskal–Wallis, Mann–Whitney, Spearman).

Null hypothesis (H₀)

The “no effect” starting assumption that a test tries to reject.

p-value

Probability of data at least this extreme if H₀ is true.

Post-hoc test

Pairwise comparisons after a significant ANOVA, adjusted for multiplicity.

Power

The probability a test detects a real effect; grows with n and effect size.

Prediction band

Range in which a new individual observation is expected to fall around a regression line (wider than the confidence band).

Q-Q plot

Sample quantiles vs theoretical normal quantiles; straight line = normal data.

R²

Fraction of Y's variation explained by a fitted model.

RCBD

Randomised complete block design — one treatment factor applied once within each block.

Residual

Observation minus its group mean (or fitted value); the “noise” a model doesn't explain.

Shapiro–Wilk test

Formal test of normality; W near 1 and p ≥ 0.05 = consistent with normal.

Slope (b)

In regression, the change in y per one-unit change in x.

Spearman correlation

Correlation of ranks; captures any monotonic relationship, resistant to outliers.

Standard error (SE)

SD ⁄ √n — the precision of a mean.

Welch's ANOVA

ANOVA variant that does not assume equal variances.

Kiwistat · KiwiScience — this guide is reachable from Help → Statistics guide in the app and from the “?” buttons next to each test. The application itself is covered in the User Guide.

↑ Back to top