Backtest literacy

A backtest runs your draft strategy over historical data and reports what it would have done. It is the main way to evaluate a change before deploying it — and the main way to fool yourself if you read it uncritically. This page is about reading one honestly. For how to run one, credits, and the report layout, see the Backtests reference.

What a backtest is — and is not

A backtest is a historical simulation and an approximation. It is not a prediction and not a guarantee. The past window it ran over will not repeat, market conditions change, and a simulation makes simplifying assumptions the live market will not honor. Treat a good backtest as evidence that a strategy is not obviously broken, not as a promise of future results.

The evidence gate

Deploying a structural change — swapping the Model, adding or removing a Filter, or publishing a brand-new strategy — asks for evidence first. The gate is deliberate, and its guiding principle is that it should never read as "you can't":

Already tested — when a completed backtest of that exact composition is linked, the Deploy button shows a "Backtested ✓" note.
Tested in a known amount of time — when a run is still needed, the answer is "run a backtest first," not a dead end. A backtest is a bounded wait, not a wall.
Deploy untested (audited) — you can always override and deploy without a backtest by giving an explicit reason, which is recorded.

(Parachute is building toward standing template backtests whose evidence a structure-identical clone can inherit — so the common case deploys with evidence already attached. That inheritance is a planned capability, not something to rely on yet.)

Adjusting a buyer dial (schedule, sizing, take-profit / stop-loss) does not change the strategy's structure and does not require a backtest.

Overfitting: the honest risk

The easiest mistake is overfitting — tuning a strategy until it looks great on one historical window, then watching it disappoint live. A strategy with many knobs can almost always be bent to fit the past; that fit is often memorizing noise, not finding a real edge. Guard against it:

Prefer settings that are robust — that work across a range of values — over a single knife-edge number that only works on one window.
Be suspicious of a result that is dramatically better than nearby settings. Real edges are usually broad; fragile ones are usually overfit.
Changing a window until the numbers improve is fitting to the window, not to the strategy.
More filters and more tuning is not automatically better. Each added knob is another way to overfit.

Read the qualifiers on every run

The report is built so nothing is hidden, and its warnings come first, above every metric. Read them before the headline numbers. The standard qualifiers:

Fills-at-ask optimism. A backtest generally assumes you get filled at the quoted price. Real fills can be worse, especially in thin books or wide spreads — so a simulated result is, if anything, an optimistic bound on execution.
Hedging and approximation annotations. Where part of the strategy — for example a hedge leg — is not fully simulated, or where a settlement had to be approximated, the report says so. A run that leaned mostly on approximation is flagged, and the deploy note carries that caveat forward.
Duration and confidence. A short window is a small sample: a few good or bad days can dominate it. The longer and more varied the window, the more the result means. A strong number over a few days is weak evidence.

A faster, cheaper look

For an instant, free preview of a threshold change — without spending a full run — use What-if replay on the canvas. It replays your candidate settings against recent recorded scans and is explicit that it shows the bot's decisions and its edge estimate at entry, not realized profit. When you need realized outcomes, run a full backtest.