Efficient Frontier and Optimal Weight Calculation With scipy

The efficient frontier is the set of portfolios that maximize expected return for each level of risk. Computing it is a constrained optimization problem — and the constraints matter more than the objective.

Setting up the inputs

You need three arrays: expected returns (mean of historical or forward estimates), the covariance matrix, and risk-free rate. Use 3–5 years of monthly returns; daily returns over-stable correlations, weekly is a reasonable middle ground.

Minimum variance portfolio

The simplest optimal portfolio minimizes variance subject to weights summing to 1:

import numpy as np
from scipy.optimize import minimize

def portfolio_var(w, cov):
    return w @ cov @ w

n = len(mu)
cons = [{"type": "eq", "fun": lambda w: np.sum(w) - 1}]
bounds = [(0, 1)] * n
w0 = np.ones(n) / n
res = minimize(portfolio_var, w0, args=(cov,), method="SLSQP",
               bounds=bounds, constraints=cons)
w_minvar = res.x

The tangency (maximum Sharpe) portfolio

The tangency portfolio maximizes the Sharpe ratio — the point where a line from the risk-free rate touches the frontier:

def neg_sharpe(w, mu, cov, rf):
    return -(w @ mu - rf) / np.sqrt(w @ cov @ w)

res = minimize(neg_sharpe, w0, args=(mu, cov, rf), method="SLSQP",
               bounds=bounds, constraints=cons)
w_tan = res.x

Tracing the frontier

For each target return level, minimize variance:

def min_var_for_target(target, mu, cov):
    cons = [{"type": "eq", "fun": lambda w: np.sum(w) - 1},
            {"type": "eq", "fun": lambda w: w @ mu - target}]
    res = minimize(portfolio_var, w0, args=(cov,), method="SLSQP",
                   bounds=bounds, constraints=cons)
    return res

Loop targets from the min-variance return to the max single-asset return to draw the curve.

Why naive optimization fails

Markowitz optimization is unstable: small input changes produce wild weight swings. Two defenses:

1. Constrain weights. Cap any single weight at 30–40% even if the optimizer wants 70%. Long-only and capped weights are far more stable than unconstrained.
2. Shrink the covariance. Replace the sample covariance with a shrinkage estimator (Ledoit-Wolf) blending toward a diagonal. This cuts estimation error dramatically.

Practical pitfalls

• Garbage returns in, garbage weights out. Mean returns are estimated with huge error; covariances are estimated more reliably. Optimize for risk first, return second.
• No transaction costs. Real rebalancing costs 10–30 bps per turn; an optimizer ignoring this churns the portfolio.
• Point estimates. The frontier is a cloud, not a line. Re-run with resampled inputs and you get a different curve each time.

The honest use

Use the optimizer to find a neighborhood of reasonable weights, then apply judgment. The tangency portfolio from raw historical data is rarely the one you should hold; the robust, constrained, shrinkage-smoothed version is closer.