Why Can Backtests Look Better Than Live Trading?
Backtests assume perfect rule following and modeled fills; live trading adds hesitation, partial fills, and distraction. Historical data is complete and clean; live data arrives with gaps and halts. You know the past; you did not know it then—subtle look-ahead creeps into signals using same-bar closes incorrectly. Strategies tuned until equity curve dazzles fit noise, not signal. Markets change liquidity, volatility, and participant behavior. A backtest is a simplified model; models err.
Treat every impressive backtest as guilty until proven robust across costs, samples, and regimes.
What Is Overfitting and How Do You Limit It?
Overfitting optimizes parameters until history is explained perfectly but future is not. Signs: many free parameters, rules that sound oddly specific, performance cliff when parameters shift slightly, great in-sample and poor out-of-sample. Limits: fewer parameters, simpler rules, walk-forward testing, penalize complexity, require economic story for each filter. If you cannot explain why a parameter value works, it probably does not. Prefer robust mediocre backtests over fragile perfect ones.
Cap optimization rounds—if round five still helps, you are mining noise, not improving edge.
What Data Biases Distort Backtest Results?
Look-ahead: using future data in signals. Survivorship bias: testing only stocks that survived—ignores delisted bankruptcies. Selection bias: choosing symbols because you know they worked. Corporate action errors: unadjusted splits inflate returns. Penny stock data quality issues. Different backtest vendors yield different results—reconcile before debating edge. Short bias: backtests assume shorts available at historical borrow rates you may not get. Document data vendor and adjustment method for reproducibility.
Run a backtest on a universe including delisted names when testing long-only historical stock picks.
How Do Regime Change and Market Structure Limit Backtests?
Zero interest rate era versus higher rates changes leadership. HFT and retail flow changed microstructure—old intraday edges decay. Volatility clustering means a strategy tested only in low VIX may fail in expansion. Regulatory changes affect shorting and leverage. COVID-era patterns may not repeat. Limitation acceptance: backtest estimates edge in past regimes similar to test period. Monitor live KPIs for decay. Plan strategy retirement rules when metrics degrade persistently.
Stress-test strategies on 2008, 2020, and 2022 subperiods—even if not your primary sample.
How Should You Use Backtests Despite Limitations?
Not as prophecy—as filter eliminating obviously bad ideas and sizing expectations. Combine with forward test, small live, and ongoing monitoring. Report uncertainty: ranges, not point estimates. Diversify strategies and timeframes modestly. Respect limitations in position sizing—do not lever to max of backtest equity curve. Humility preserves capital when the model breaks.
Write a limitations paragraph beside every backtest summary—future you will need the reminder.