Backtests, honestly

Why we expect our live results to be lower than our backtests

A backtest shows what a set of rules would have done. Ours was built with care: it charges realistic trading costs and fund fees, and it shows every figure as a range across the different days of the month a model could trade. But it is not a track record, and we expect real results to come in lower, for four reasons.

  1. 1

    We designed it with hindsight.

    We chose the rules over the same 20 years we test them on. Even simple rules, picked from many we tried, look better on the history they were picked on.

  2. 2

    The history is missing the funds that closed.

    Our data does only cover funds that still exist, so there is a survivorship bias risk. We reviewed the history of funds that closed, and found they were overwhelmingly low-capitalization ones our rules would have skipped, so we expect this effect to be small. (We did measure a related effect, choosing the fund menu with hindsight: about a tenth of a point a year.)

  3. 3

    Costs are estimates.

    We charge realistic trading costs and fund fees, but real prices, spreads and timing differ, more so for leveraged ETFs.

  4. 4

    The past never perfectly predicts the future.

    We've tested across a wide range of markets: booms, crashes, rate shocks, long bull runs. But no two markets are the same, and the next one will differ in ways no backtest can anticipate. What we do expect to last is the core phenomenon. Momentum has persisted for centuries because it's rooted in human nature: investors under-react to news, then pile in late.

What we do about it

Every monthly signal is signed and timestamped before the month starts, so our live record can't be edited after the fact (the signed signals). We publish it separately from the backtest, and as it grows, it replaces the backtest as the number we lead with. More hard questions, answered directly: FAQ.

Trade signals free through the December 1 reallocation — subscribe now, trade today.