Walk-Forward and the Million-Dollar Questions

In February 2015 someone asked on the QuantConnect forum whether walk-forward optimisation could be added. The founder replied, warmly. The thread kept resurfacing over the years, and when somebody dug it up again, the answer was worth quoting verbatim: "Officially scheduled for 2023."

In March 2023 a GitHub issue duly appeared. It is still open, and its most recent comment is from April this year.

Eleven years since that first request.

I don't think that's laziness. I think it's because walk-forward isn't a feature you add. It's a rhythm — and a rhythm doesn't fit inside a button, because setting one up means answering four questions nobody has a universal answer to. Those four are what this piece is about.

The most-cited discipline nobody agrees on

Walk-forward is not new and it certainly isn't our idea. I'd say it's the single most-cited discipline in algorithmic trading. It's also one where two people almost never agree on the actual settings.

Not because one of them is an idiot. Because the right settings depend on what you trade, how long you hold, and how your risk management is built. And because everyone guards the strategies that actually work, so as a matter of principle they can't converge. Anyone who claims to know the settings that always fit is lying.

A strategy holding positions for weeks and one holding them for two hours simply don't run at the same tempo. And watch the trade counts too, in the in-sample window and in the out-of-sample one alike. Passing off a handful of trades as a statistic is closer to a joke than to honest work.

What isn't in dispute: walk-forward is the cheapest defence against curve-fitting, and used well it leads to settings that make money. It isn't immunity — you can overfit the walk-forward itself, by playing with window lengths until a pretty picture falls out. But it's the cheapest and fastest defence available.

There's one more thing that gets talked about less: it reliably exposes junk. Strategies that looked good in a backtest and come apart in the first honest out-of-sample window. Those have no place with us, and walk-forward is the quickest way to spot them before you spend real time on them.

Junk needs catching at the start, without sentiment — no "maybe there's an edge in there somewhere" or "what if I just didn't test it enough and this was the one". Better to spend the time on strategies that show promise immediately.

Why doing it once isn't enough

The usual picture goes like this: run a walk-forward, it passes, done, go live.

But the test doesn't validate the one parameter set you walk away with. It validates the ability to re-fit — and whether the metrics you watch in-sample carry over into out-of-sample at all. It tests a process: take a slice of history, find settings, run them on a slice you haven't seen, step forward, repeat. Survive twenty rounds of that and you have reason to believe the twenty-first will hold too. More to the point, you've found a process you now have to sustain in live trading. Re-fitting becomes part of the strategy rather than an exceptional event — you re-fit on the rhythm of the window that passed the test.

Pass that test and then never re-fit again, and you've invalidated your own evidence. You tested a process you don't use. You're holding settings from a single window while pretending you have twenty behind you.

Walk-forward isn't a rubber stamp. It's a rhythm.

We've said as much on this site before, in a more general piece on walk-forward. This one is about what follows — because once you accept it's a rhythm, those four questions stop being dull configuration details and become the parameters of that rhythm.

How long should the rhythm be

Everyone wants a number here, and the honest answer is that no universal number exists.

The freshest trace in that eleven-year-old thread is from April this year. One of the participants argues that window length should be chosen by cross-validation rather than fixed, and adds that six months in-sample against one month out-of-sample works for him on daily data — while flagging straight away that it depends on holding period. I read that as one practitioner's experience, not a recommendation, and so should anyone else.

The same decision covers anchored versus rolling windows. An anchored window keeps everything from the start; a rolling one forgets. That isn't a technical detail, it's a decision about how much memory the system should have — whether a crisis from ten years ago should still concern it. You can of course run more tests, and robustness will come knocking to thank you for it.

How you know the strategy has fallen out of rhythm

Here's the part people work on least and that decides the most.

The question of degradation metrics comes round again and again on the forums, and the answers don't line up. Without one, the whole rhythm is blind: you re-fit at regular intervals without knowing whether you're rescuing anything or just repainting a rusty fence on schedule.

The difference between "that was a weaker month" and "this strategy is finished" has to be decidable in advance. Not at the moment you're staring at it and rather fancy the first reading.

The analysis gives you plenty to work with: which out-of-sample window came out of which in-sample one, how readily the parameters carried between windows, and what results you can therefore expect. That's not nothing — that's the basis for a rule.

And to stop trading turning into gambling, the condition for switching a strategy off has to be set in advance too. Usually it's a logical or — one of several things is enough.

Degradation metrics aren't the only safeguard, either. A strategy needs risk management, above all a hard cap on the loss from any single position, plus other protective mechanisms. I want to write a separate, more technical piece on that, including how risk management ties into walk-forward and degradation metrics.

What you're optimising for on each re-fit

Questions about the objective function tend to vanish from forums as unclear or duplicate. That's telling in itself — people sense the decision exists but can't name it well enough to keep a thread alive.

On GitHub someone put it precisely, in May 2024: he wanted to pass in his own objective function, one that could reach the runtime statistics of the whole backtest rather than a single point estimate. That's the heart of it. Optimise for one number and you get the settings that are best at that one number, while everything you didn't declare an interest in quietly drifts.

Worth mentioning one thing from our own kitchen here, because it bears directly on all of this. We do use cross-validation, just for a different question than the one that April commenter had in mind. In its combinatorially symmetric form, as described by Bailey, Borwein, López de Prado and Zhu, it estimates the probability that a result is overfit — synthesising out-of-sample from a single run, so it works even where there is no walk-forward yet. We use it as a first sieve on genetic and grid runs, before windows come into it at all.

That sounds like a digression. It isn't. It shows that the four questions aren't four independent decisions but one system — and the order you settle them in changes what the test can tell you in the first place.

And when it stops working

There's a long thread on EliteTrader whose title asks whether walk-forward isn't simply an illusion. Somewhere in it, the question comes up of how long it held for anyone in live trading. No answer. Not because it's a secret — because nobody tracks it systematically.

My view: an edge doesn't fade. It dies. And the protection against that isn't another test, it's bet size and the willingness to switch the thing off.

So the re-fitting rhythm needs a boundary, past which you stop adjusting and stop altogether.

In closing

That eleven-year-old request isn't about a missing feature. It's about people wanting someone to answer those four questions honestly and say out loud why that way. That isn't a button, that's a stance.

And part of that stance is one more answer you have to give yourself: when you stop re-fitting. Without it you'll keep correcting downwards indefinitely, telling yourself the whole time that the market is simply in a different regime.

Sometimes it is. And sometimes it's over.

Want a strategy that re-fits on the rhythm that passed the test — and knows when to stop? Get in touch →