Sample size calculator (power analysis)

t tests (exact). Under the alternative, the t statistic follows a noncentral t distribution with $\nu$ degrees of freedom and noncentrality $\delta$. For two independent groups $\nu =n_1+n_2-2$ and $\delta =d\cdot \sqrt{n_1{n}_2/(n_1+n_2)}$, where $d=({\mu }_1-{\mu }_2)/\sigma$; for a paired (or one-sample) test $\nu =n-1$ and $\delta =d_z\cdot \sqrt{n}$, where $d_z$ is the mean difference divided by the SD of the differences. Two-sided power is

$$\begin{array}{rl}1-\beta & =1-F_t(t_{1-\alpha /2,\nu };\nu ,\delta )\\ & +F_t(-t_{1-\alpha /2,\nu };\nu ,\delta )\end{array}$$

and one-sided power drops the last term and uses $t_{1-\alpha ,\nu }$. The noncentral t distribution function is computed with Lenth’s algorithm AS 243 (as in R’s pt). When entering means and SDs, $d$ uses the pooled SD $\sqrt{({\mathrm{SD}}_1^2+{\mathrm{SD}}_2^2)/2}$.

One-way ANOVA. With $k$ groups of $n$, the F statistic follows a noncentral F distribution with $k-1$ and $k(n-1)$ degrees of freedom and noncentrality $\lambda =f^2\cdot k\cdot n$, where Cohen’s $f$ is the SD of the group means (divided by $k$) over the within-group SD.

$$\text{Power}=1-F_F(F_{1-\alpha };k-1,k(n-1),\lambda )$$

computed as a Poisson-weighted sum of incomplete beta functions. This is the convention of G*Power and R’s pwr.anova.test.

Two proportions. The normal approximation with the pooled proportion $\overline{p}=(p_1+k\cdot p_2)/(1+k)$ under $H_0$ and separate variances under $H_1$ ($n_2=k\cdot n_1$):

$$\begin{array}{rl}{n}_1=[& z_{1-\alpha /2}\sqrt{\overline{p}\overline{q}(1+1/k)}\\ & +z_{1-\beta }\sqrt{p_1{q}_1+p_2{q}_2/k}{]}^2\\ & /(p_1-p_2{)}^2\end{array}$$

The optional continuity correction is that of Fleiss, Tytun and Ury (1980):

$$n^{\prime }=\frac{n}{4}{(1+\sqrt{1+\frac{2(k+1)}{n\cdot k\cdot |p_1-p_2|}})}^2$$

Correlation. The Fisher z approximation used by R’s pwr.r.test: the critical $r$ is found from the t distribution with $n-2$ degrees of freedom, and $\text{power}=\mathrm{\Phi }((z_{\rho }-z_{r\mathrm{crit}})\sqrt{n-3})$ (plus the lower tail when two-sided), with $z_{\rho }=\mathrm{atanh}\rho +\rho /(2(n-1))$.

Sample size is the smallest whole $n$ whose power reaches the target, found by search, so it already accounts for rounding and for the allocation ratio ($n_2=\lceil k\cdot n_1\rceil$). Cohen’s conventional effect sizes: $d=0.2,0.5,0.8$; $f=0.10,0.25,0.40$; $\rho =0.1,0.3,0.5$ (small, medium, large). They are a last resort: an effect estimated from pilot data or the smallest effect that matters is better.

Checks. The distribution functions agree with SciPy’s nct and ncf to at least 10 significant figures. $d=0.5$ gives 64 per group (G*Power and pwr.t.test: 63.77 rounded up), $d_z=0.5$ gives 34 pairs, $f=0.25$ with 3 groups gives 53 per group (159 in total), and $\rho =0.3$ gives 85 (pwr.r.test: 84.07 rounded up).

Sources: Cohen J. Statistical Power Analysis for the Behavioral Sciences, 2nd ed. Hillsdale, NJ: Lawrence Erlbaum; 1988 (reissued by Routledge, doi:10.4324/9780203771587). Faul F, Erdfelder E, Lang A-G, Buchner A. G*Power 3: a flexible statistical power analysis program for the social, behavioral, and biomedical sciences. Behav Res Methods 2007;39:175–191, doi:10.3758/BF03193146. Lenth RV. Algorithm AS 243: cumulative distribution function of the non-central t distribution. Appl Stat 1989;38:185–189, doi:10.2307/2347693. Fleiss JL, Tytun A, Ury HK. A simple approximation for calculating sample sizes for comparing independent proportions. Biometrics 1980;36:343–346, doi:10.2307/2529990.

Two-sample t test. You expect treated cells to differ from controls by half an SD ($d=0.5$) and will test two-sided at $\alpha =0.05$ with 80% power. With 64 per group, $\nu =126$, the critical $t$ is 1.9790 and $\delta =0.5\times \sqrt{64\times 64/128}=2.828$, giving power 0.8015; with 63 per group power is 0.7952, so 64 per group (128 in total) is the answer. Entering means 10 and 12 with SDs of 4 gives the same $d=2/4=0.5$.

• One-sided instead: 51 per group.
• Power 0.9 instead of 0.8: 86 per group.
• Twice as many in group 2 (ratio 2): $n_1=48$, $n_2=96$ (power 0.8021).
• With only 20 per group, power is 33.8%.

One-way ANOVA. Three groups with $f=0.25$: with 53 per group ($N=159$) the F test has 2 and 156 degrees of freedom, critical $F=3.054$ and $\lambda ={0.25}^2\times 159=9.94$, so power is 0.8049 (52 per group gives 0.7967). Entering group means 10, 12 and 14 with a within-group SD of 6 gives $f=1.633/6=0.272$.

Two proportions. A response rate of 65% with the standard treatment and 85% expected with the new one, $\alpha =0.05$ two-sided, power 0.8: $n=72.4$, so 73 per group (power 0.8033). With the Fleiss continuity correction it is 83 per group.