Skip to content
Explore Automated Analysis

How to calculate power for an independent-samples t-test in Statsmodels (with Code)

Editorial · Published

The Short Answer (TL;DR)

Use TTestIndPower.solve_power with a planned standardized mean difference, significance level, target power and group allocation ratio to solve for the first group size. Round the group sizes to whole observations and recompute achieved power with their actual ratio for the pooled equal-variance t-test design.

from statsmodels.stats.power import TTestIndPower
n1 = TTestIndPower().solve_power(effect_size=.5,alpha=.05,power=.8,ratio=1.5,alternative="two-sided")
print(n1,1.5*n1)

Detailed Implementation

This example uses synthetic data and does not need a network connection. Install the recorded package versions in a separate Python environment before running it.

python -m pip install numpy==2.5.4 scipy==1.18.1 pandas==3.0.6 statsmodels==0.15.0
from pathlib import Path
from math import ceil,isfinite
from statsmodels.stats.power import TTestIndPower

def plan_two_sample_t(effect_size, alpha, target_power, ratio):
    values=(effect_size,alpha,target_power,ratio)
    if not all(isfinite(v) for v in values):
        raise ValueError("All planning inputs must be finite.")
    if effect_size <= 0 or ratio <= 0 or not 0 < alpha < target_power < 1:
        raise ValueError("Require positive effect and ratio, with 0 < alpha < target power < 1.")
    design=TTestIndPower()
    n1_real=design.solve_power(effect_size=effect_size,alpha=alpha,
                               power=target_power,ratio=ratio,alternative="two-sided")
    n2_real=ratio*n1_real
    n1,n2=max(2,ceil(n1_real)),max(2,ceil(n2_real))
    actual_ratio=n2/n1
    achieved=design.power(effect_size=effect_size,nobs1=n1,alpha=alpha,
                          ratio=actual_ratio,alternative="two-sided")
    if not isfinite(achieved) or achieved < target_power-1e-8:
        raise ValueError("Rounded allocation did not meet target; revise the design.")
    return n1_real,n2_real,n1,n2,achieved

# Illustrative planning inputs, chosen before collecting outcomes.
d,alpha,target_power,ratio=.5,.05,.8,1.5
n1_real,n2_real,n1,n2,achieved=plan_two_sample_t(d,alpha,target_power,ratio)
report=(
    f"continuous solution: n1={n1_real:.6f}, n2={n2_real:.6f}\n"
    f"rounded allocation: n1={n1}, n2={n2}, total={n1+n2}, ratio={n2/n1:.9f}\n"
    f"planned d={d}; two-sided alpha={alpha}; achieved power={achieved:.9f}\n"
)
Path("independent-t-power.txt").write_text(report,encoding="utf-8")
print(report)

The continuous group sizes are approximately 53.105061 and 79.657592; rounding gives 54 and 80 analyzable observations, total 134, with achieved two-sided power 0.804633326.

What does the standardized effect represent?

The positive planning effect d is the magnitude of a mean difference divided by a common within-group population standard deviation. The value 0.5 here is illustrative, not a universal recommendation or an estimate of what a future experiment will observe. Choose a scientifically meaningful difference and a defensible variability assumption before collecting outcomes; a sensitivity analysis over plausible effects is often more informative than one unsupported point assumption.

Which group size and ratio are being solved?

The continuous solution is about 53.105 observations in group 1 and 79.658 in group 2, using ratio=n2/n1=1.5. Observations are indivisible, so this example rounds each continuous group size upward, obtaining 54 and 80. Their actual ratio is about 1.48148. Recompute power with that ratio instead of silently assuming rounding preserved the original allocation exactly.

What does the power calculation assume?

This design uses a two-sided pooled two-sample t-test, independent observations, a common population variance and normal-reference model assumptions. It does not automatically plan an unequal-variance Welch test, a paired test or clustered data. Plan for the analysis that will actually be used. The output counts are analyzable observations; incomplete outcomes or unequal retention require a separately justified recruitment plan.

What is independently checked?

For the rounded allocation, the validation calculates both noncentral-t rejection tails directly, with 132 degrees of freedom and noncentrality d/sqrt(1/n1+1/n2). That probability agrees with achieved power of about 0.804633. Swapping the group sizes preserves two-sided power, and a zero effect gives the type-I-error probability. Larger planned effects need fewer observations, while a higher power target needs more in the checked scenarios. Power is conditional on the chosen alternative and assumptions, not a guarantee that a study will reject or an informative substitute for its confidence interval.

Mathematical Context

For integer group sizes n1 and n2, the pooled t statistic has degrees of freedom nu=n1+n2-2. Under standardized mean difference d, its noncentrality is lambda=d/sqrt(1/n1+1/n2). Let c be the 1-alpha/2 quantile of the central t distribution with nu degrees of freedom. Two-sided power is P(T>c)+P(T<-c) for T following the corresponding noncentral t distribution. The example gives n1=54, n2=80, nu=132 and power=0.804633326. Effect size, alpha, allocation ratio and power are dimensionless; sample sizes count observations. An observed post-study effect is not a reliable basis for claiming a study had a prespecified design power.

\[ \lambda=\frac{d}{\sqrt{1/n_1+1/n_2}},\qquad \mathrm{Power}=P(T_{\nu,\lambda}>c)+P(T_{\nu,\lambda}<-c) \]

Reproducibility

Executed with Python 3.12.14; numpy 2.5.4, scipy 1.18.1, pandas 3.0.6, statsmodels 0.15.0. Automated checks confirm the stated output properties.

Primary documentation

Related guides