Confidence Intervals and Trading Uncertainty

A backtest says your strategy returns 12% a year. The honest version says: "between −3% and 27%."

Any single estimate — a win rate, a Sharpe ratio, an expected return — is a point guess drawn from a noisy sample. Confidence intervals (CIs) wrap that point in the range it could plausibly occupy, telling you how much to trust the number.

The formula

For a mean estimated from a sample of size n:

CI = x̄ ± z · (s ÷ √n)

Where z is the critical value (1.96 for a 95% CI under the normal approximation), s the sample standard deviation, and √n the standard-error shrinkage factor.

Example: 100 trades, mean $50 per trade, σ = $200.

CI = 50 ± 1.96 · (200 ÷ 10) = 50 ± 39.2 → [$10.8, $89.2]

So the true mean per trade is plausibly between $11 and $89 — and might be lower. The point estimate alone hides this range entirely.

Why this matters for traders

1. Strategy comparison: if Strategy A's 95% CI is [$1, $99] and Strategy B's is [$30, $70], B is more reliable even with a similar mean — its uncertainty band is tighter
2. Risk of ruin: a CI that straddles zero means you can't rule out a money-losing strategy
3. Position sizing: size off the lower bound of your edge's CI, not the point estimate — a conservative, robust choice
4. Sample size design: before backtesting, use the formula to decide how many trades you need for a CI narrow enough to act on

What a confidence interval is not

A 95% CI does not mean "95% probability the true mean is in this range." The true mean is fixed; it's either in the interval or not. The 95% refers to the long-run frequency: if you repeated the experiment many times, 95% of the resulting intervals would contain the true mean.

This distinction matters when markets are non-stationary — the "true" mean keeps moving, so your CI is only as good as the assumption that the regime holds.

Practical workflow

1. Compute mean and σ of your per-trade results
2. Build a 95% CI around the mean
3. If the lower bound is negative, your "edge" is statistically indistinguishable from noise
4. Increase sample size or reduce variance before scaling capital
5. Re-compute the CI live; if it shifts to include zero, the edge may have decayed

Summary

A single backtest number is a headline, not a fact. Confidence intervals force you to confront how wide the plausible range of outcomes really is. Trade the lower bound of your confidence, not the point estimate — and you'll size conservatively enough to survive the days when reality falls at the bad end of the interval.