How to calculate a Spearman rank correlation in SciPy (with Code)
The Short Answer (TL;DR)
Use scipy.stats.spearmanr to calculate the correlation of average ranks, keeping each pair of observations aligned. For the small sample here, the full implementation enumerates all pairings to obtain a permutation p-value instead of relying on the routine asymptotic approximation.
from scipy import stats
x, y = [1,2,2,4,5,6], [3,1,2,5,4,6]
print(stats.spearmanr(x, y, nan_policy="raise", alternative="two-sided").statistic)
Detailed Implementation
This example uses synthetic data and does not need a network connection. Install the recorded package versions in a separate Python environment before running it.
python -m pip install numpy==2.5.4 scipy==1.18.1
from pathlib import Path
import numpy as np
from scipy import stats
x = np.array([1, 2, 2, 4, 5, 6], dtype=float)
y = np.array([3, 1, 2, 5, 4, 6], dtype=float)
if x.ndim != 1 or y.ndim != 1 or x.size != y.size or x.size < 3:
raise ValueError("Use matched one-dimensional arrays with at least three pairs.")
if not (np.isfinite(x).all() and np.isfinite(y).all()):
raise ValueError("Resolve non-finite observations jointly before ranking.")
if np.ptp(x) == 0 or np.ptp(y) == 0:
raise ValueError("Spearman correlation is undefined for a constant input.")
if x.size > 8:
raise ValueError("This exhaustive example is restricted to at most eight pairs.")
result = stats.spearmanr(x, y, nan_policy="raise", alternative="two-sided")
rank_x = stats.rankdata(x, method="average")
rank_y = stats.rankdata(y, method="average")
centered_y = rank_y-rank_y.mean()
denominator = np.sqrt(np.sum((rank_x-rank_x.mean())**2)*np.sum(centered_y**2))
def rank_correlation(permuted_ranks):
return np.dot(permuted_ranks-permuted_ranks.mean(),centered_y)/denominator
# Fix y and permute x's labelled ranks under an exchangeable-pairing null.
permutation = stats.permutation_test(
(rank_x,), rank_correlation, permutation_type="pairings",
vectorized=False, n_resamples=np.inf, batch=120, alternative="two-sided")
report = (
f"Independent matched pairs: {x.size}\n"
f"Average ranks of x: {rank_x.tolist()}\n"
f"Spearman rho: {result.statistic:.8f}\n"
f"All labelled pairings enumerated: {permutation.null_distribution.size}\n"
f"Exact two-sided pairing p-value: {permutation.pvalue:.8g}\n"
f"Asymptotic p-value (comparison only): {result.pvalue:.8g}\n"
)
print(report)
Path("spearman-correlation-output.txt").write_text(report, encoding="utf-8")
The report shows average x ranks [1, 2.5, 2.5, 4, 5, 6], rho approximately 0.75370235, 720 enumerated pairings and exact two-sided p=0.11111111. It separately labels the asymptotic comparison p approximately 0.08352328; the short answer prints only the coefficient.
How are tied values ranked?
The two occurrences of two in x occupy rank positions two and three, so both receive the average rank 2.5. Spearman rho is Pearson correlation applied to these rank vectors. The familiar formula using squared rank differences without a tie adjustment is inappropriate here. Strictly increasing transformations preserve the ordering and ties, so they preserve the coefficient.
Why enumerate pairings for this small sample?
SciPy cautions that its usual p-value approximation is accurate only for very large samples. This six-pair example enumerates 720 labelled permutations of one rank vector while the other remains fixed. Ties produce repeated rank arrangements; each distinct arrangement has the same multiplicity here, and the labelled enumeration handles that multiplicity explicitly. Do not attempt factorial enumeration for a large dataset.
Which null does the permutation calculation need?
The calculation assumes independent units and that the pairing of x and y is exchangeable under the null of no association. Zero rank correlation alone does not imply independence, so arbitrary dependent structures are not justified by that null description. Time series, clusters and repeated observations can violate exchangeability. The two-sided probability uses twice the smaller inclusive tail, capped at one; state that convention when comparing software.
Mathematical Context
Let R_i and S_i be the average ranks within the two variables. Compute their centered cross-product divided by the square root of the product of their centered sums of squares. The six-pair coefficient is approximately 0.753702. Under the exchangeable-pairing null, hold S fixed and permute the six labelled R values in 6!=720 ways. The independent validation compares the rank-product numerators directly, counts each inclusive tail, and doubles the smaller proportion, capped at one. This produces p=1/9, approximately 0.111111, whereas the routine asymptotic comparison is approximately 0.083523. Neither p-value is a posterior probability of the null, and rho describes monotonic association rather than causality.
\[ \rho_s=\frac{\sum_i(R_i-\bar{R})(S_i-\bar{S})}{\sqrt{\sum_i(R_i-\bar{R})^2\sum_i(S_i-\bar{S})^2}} \]
Reproducibility
Executed with Python 3.12.14; numpy 2.5.4, scipy 1.18.1. Automated checks confirm the stated output properties.
Primary documentation
- SciPy Spearman correlation and small-sample guidance
- SciPy pairing permutations and inclusive two-sided tails
- SciPy average ranks for ties