What Questions Should Strategy Evaluation Answer?
Does it make economic sense? Enough trades for statistics? Profitable after costs on in-sample and out-of-sample? Drawdown survivable? Stable across nearby parameters? Works in multiple years, not one? Live KPIs tracking backtest? Executable without rule violations? Correlated with other strategies you run? If any answer fails, status is research—not live capital. Evaluation is gatekeeping, not cheerleading.
Write pass/fail criteria before viewing results—prevents moving goalposts after you see equity curves.
How Do You Test Robustness Beyond One Backtest?
Parameter sensitivity: small changes should not zero edge. Monte carlo trade reorder and slippage stress. Different universes—large cap only versus mid cap. Subperiod tests: bull, bear, chop years. Walk-forward aggregate. Paper trading slippage log. Compare to naive benchmark. Robust strategies show degraded but positive under stress; fragile ones vanish. Robustness beats peak backtest profit for career longevity.
Include a deliberately harsh cost scenario—if edge survives double slippage, sizing confidence improves.
How Much Live Data Do You Need to Confirm?
Minimum thirty to fifty trades for initial comparison; one hundred for confidence on win rate and profit factor. Compare rolling metrics, not all-time after trade five. Tag regime. Investigate divergence: execution, rule drift, sample luck, or edge gone. If live underperforms after honest audit, reduce size or pause—not hope. Confirmation takes time; impatience deploys unproven size.
Match live sample length to typical hold period—thirty day trades mean months; thirty scalps mean weeks.
When Should You Retire or Revise a Strategy?
Retire: out-of-sample and live fail, economic story broken, structural market change, repeated rule violations you cannot fix. Revise: minor filter added with new backtest version, sizing change, session window tweak—never mid-loss streak without data. Retirement is success—capital preserved for better edge. Revision requires version control like software. Evaluation ongoing monthly, not once.
Retiring a strategy should trigger a written post-mortem stored with its backtest files for future reference.
How Does Evaluation Fit the Development Cycle?
Develop, backtest, evaluate, paper, evaluate, small live, evaluate, scale or stop. OddsMaker and similar tools accelerate middle steps but do not remove judgment. Evaluation checklist in trading plan appendix. Peer review if available. Document decisions—scale, hold, cut—for learning. Serious traders have a graveyard of tested-no strategies; that is feature, not failure.
Keep a rejected-strategies log—prevents retesting the same dead idea eighteen months later.
Evaluation quality improves when you score each strategy on a simple one-to-five robustness rubric before comparing P&L.
Share evaluation summaries with a peer when possible—blind spots in your own backtests are easier for others to spot.