Regression analysis models relationships between variables — for example, how much a stock's return is explained by market returns, interest rates, or sector performance. Multiple regression can separate the contribution of each factor, helping isolate what truly drives returns.
Cointegration identifies pairs of securities that share a long-term equilibrium relationship, even if they diverge in the short term. This is the mathematical foundation of pairs trading and statistical arbitrage.
These tools are powerful, but in markets they come with a warning label. Financial data is noisy, limited, and produced by people who adapt. The same regression that works beautifully on physics data can confidently find patterns in market data that were never really there. Most of this chapter is about that warning label.
Correlation measures how two things move together, on a scale from minus 1 to plus 1. It says nothing about why.
When a signal correlates with future returns, there are at least four possible stories. The signal genuinely causes the returns. The returns cause the signal (reverse causality). Some third factor drives both — small stocks might look magical when the real driver is their extra risk. Or it's pure coincidence, which gets likelier the more series you compare. A famous illustration: an economist once found that Bangladeshi butter production tracked the S&P 500 remarkably well. Search enough data series and something absurd will always fit.
Why care, if the pattern trades profitably either way? Because only causal relationships give you a reason to expect the pattern to persist. A coincidental correlation can vanish without warning, and a third-factor correlation breaks the moment the third factor shifts. Before trusting a signal, ask the childlike question: why would this work? If nobody can tell a plausible economic story — someone mispricing something for an understandable reason — be suspicious of the fit.
A p-value estimates how likely your result would appear by pure chance if no real effect existed. The convention treats results below 5 percent as statistically significant. That threshold has a hidden cost: test twenty worthless strategies and, on average, one clears the bar by luck alone.
Now scale that up. A researcher with a backtesting tool can test hundreds of variations in an afternoon — different lookback windows, thresholds, universes, date ranges. Keep the best and you've practically guaranteed an impressive-looking fluke. Doing this — searching until something clears significance and reporting only that — is called p-hacking, and in strategy research it usually happens innocently. Nobody thinks of tinkering as running hundreds of experiments. Statistically, that's exactly what it is.
The defenses are simple and a little painful. Count everything you tried, not just the winner, and demand stronger evidence the more you searched. Hold out data your tinkering never touched, and let the strategy face it exactly once. And prefer ideas you specified before looking. Finance researchers Harvey, Liu, and Zhu argued that because of this problem, most claimed findings in empirical finance are likely false.
Nearly every statistical tool assumes stationarity — the property that a data series keeps the same basic statistical character over time, with a stable average and a stable amount of wiggle. Coin flips are stationary: the odds never drift. Markets aren't so obliging.
Financial series shift regimes. Volatility comes in storms and calms. Correlations that held for a decade can flip in a crisis — infamously, in a crash, everything falls together, right when diversification was supposed to help. Interest rate behavior in the 2010s looked nothing like the 1970s. When you fit a model to non-stationary data as if it were stationary, you're computing precise answers about a market that no longer exists.
This is also the deep reason cointegration tests exist: they check whether a relationship between two prices is stable enough to bet on, rather than assuming it. Practical habits help more than fancy fixes. Test strategies across separate decades and regimes rather than one long blended average. Ask of every input: is this stable enough to lean on? And hold conclusions loosely, because the ground under any market model moves.
Monte Carlo simulation generates thousands of random scenarios to model uncertainty and estimate the probability distribution of portfolio outcomes. It helps answer questions like: What is the probability of losing more than 20% in a given year?
Machine learning methods — random forests, gradient boosting, neural networks — can identify non-linear patterns in financial data that traditional statistical methods miss. However, financial data is notoriously noisy, and ML models are prone to overfitting without careful validation.
Everything in this chapter applies double to machine learning. A neural network is a pattern-finding machine of enormous flexibility, pointed at data that is mostly noise, shaped by regimes that don't repeat. It will find something; the question is whether that something is real. Simpler models with fewer knobs are often the wiser choice in finance precisely because they can't contort themselves around noise the way flexible ones can.
Researchers have catalogued hundreds of market anomalies — patterns that seem to beat the market, from calendar effects to accounting signals. Follow-up work keeps reaching the same sobering conclusion: most of them shrink or vanish after publication. McLean and Pontiff, tracking nearly a hundred published anomalies, found returns dropped substantially out of sample and decayed further once the papers were public.
Two forces do the damage, and it matters which one you're facing. The first is that the anomaly was never fully real — a product of multiple comparisons, data mining, or a particular non-stationary regime. Those findings fail immediately when the luck runs out. The second is arbitrage: the finding was real, but publication invited capital to trade it, and the crowd's own buying competed the edge away. Those decay gradually.
Either way, the practical stance is the same. Treat any historical pattern — from a journal, a backtest, or this site — as a decaying asset with an unknown half-life. Ask when it was discovered, how easily it's traded, and how crowded it's become. Some premiums have persisted for decades. But persistence is something a pattern has to keep proving, and no backtest can prove it in advance.
Previous: Senate Trading Analysis · Next: Compound Interest: The Eighth Wonder · Financial glossary
← Back to all investing concepts