MyAlgoGuru Help
Education Glossary Open the app
Chapter 1 of 5

What a backtest is actually telling you

A backtest answers one narrow question very well, and several important questions not at all. Knowing which is which is most of the skill.

A backtest answers exactly one question:

"If these rules had been followed mechanically over this period, on this data, what would have happened?"

Read that sentence again, because every word in it is load-bearing. These rules. This period. This data. Mechanically. Change any one and the answer changes, sometimes completely.

What it does not answer is whether the strategy is any good. That is a different question, and no single test can settle it.

The four gaps

Between the number your test reports and the number your broker statement shows, there are four gaps. They are not mysterious, and three of the four are fixable.

Backtest what the test says Live what you actually got Costs fixable brokerage, STT, stamp and GST Slippage estimable the fill you got, not the one asked Data fixable gaps, look-ahead, dead contracts You not fixable the rule you overrode at 2pm Three of the four are yours to close. The fourth is a discipline problem, and no backtester has ever solved one.
The four gaps. A backtest models a market. It does not model your broker's charges, the depth of the book, the holes in your data, or you.

The first three are the subject of this module. The fourth is the subject of your trading journal.

Mechanically means mechanically

The word doing the most quiet work in that opening sentence is mechanically.

A backtest follows your rules with no judgement at all. It does not skip a signal because the day felt wrong. It does not widen a stop because the move looked like a wick. It does not sit out the session after the Budget speech because the tape was chaotic.

This is the backtest's great strength and its great deception at once. Its strength, because a mechanical result is reproducible and you can reason about it. Its deception, because you are not mechanical, and the thing you will actually trade is you plus the rules, not the rules alone.

Tip

If you have ever looked at a backtest and thought "I would not have taken that one", you have just measured the fourth gap. Write down how many trades you would have skipped. Then run the test again without them and see whether the strategy survives. Often what you are calling judgement is the only thing holding the curve up, and that is worth knowing before you automate it.

Gross is a number. Net is the trade.

The single most common way a test flatters you is by reporting gross.

Consider a modest intraday strategy: one NIFTY options round trip a day, twenty sessions a month. That is forty orders a month, four hundred and eighty a year. Each one carries brokerage, STT, exchange transaction charges, SEBI turnover fees, stamp duty and GST.

Worked example

Say your rules produce an average gain of ₹420 per round trip before costs, and your all-in cost per round trip is ₹95.

  • Gross, twenty trades a month: 20 × ₹420 = ₹8,400
  • Costs: 20 × ₹95 = ₹1,900
  • Net: ₹6,500

That is a 23 percent haircut, and the strategy still works. Now halve the edge, because you tightened a filter and took fewer, smaller trades:

  • Gross: 20 × ₹210 = ₹4,200
  • Costs: unchanged at ₹1,900
  • Net: ₹2,300

Halving the edge did not halve the result. It cut it by nearly two thirds, because costs do not shrink when your edge does. Cost is a fixed tax on activity, not a percentage of success.

Figures here are arithmetic for illustration. They are not a result from any strategy, and nothing here suggests what any strategy will earn.

This is why MyAlgoGuru reports backtests gross by default and applies charges explicitly. Not to flatter the number, but the opposite: so that you always know which of the two you are looking at, and have to make a deliberate choice to see the other. A tool that silently nets costs is a tool whose assumptions you cannot inspect.

The module What a trade actually costs takes that stack apart line by line, including the one mistake that makes an options backtest wrong by more than a factor of twenty.

Look-ahead: the bar you could not have had

The second gap is subtler and much more dangerous, because it does not make your result slightly worse. It makes it fictional.

Look-ahead bias is using information at a point in time when you could not have had it. The classic version in a bar-based test: deciding to enter on a bar using that same bar's close.

What the test did entered here, at this bar's close signal bar The close is only known once the bar is over. At the moment of decision, that price did not exist yet. What you could have done entered here signal bar next bar The bar closes, the signal is confirmed, you act on the next open. Slower, and real.
Look-ahead in one picture. On the left, the test enters at a price that was not knowable until the bar had already finished. On the right, the signal is confirmed at the close and acted on at the next open. The second is worse on paper and achievable in life.

A strategy that enters at the signal bar's close is buying with tomorrow's newspaper. The effect is not small. On a volatile instrument it can turn a losing set of rules into a beautiful curve, because you are systematically entering at a price you selected because of how the bar ended.

Warning

Look-ahead is the one bias that does not degrade gracefully. Costs make a good strategy slightly worse. Look-ahead can make a worthless strategy look excellent, and you will not find out until real money is in it.

The same mistake wears other clothes. Using a daily indicator value intraday before the day has closed. Using an adjusted price series that was adjusted after the fact. Filtering your universe by something that was only true later, which brings us to the third gap.

Survivorship: the contracts that are not there any more

Ask your tool to backtest "the F&O stocks" over three years and it will quietly hand you today's F&O list, tested backwards.

That list is not the list that existed three years ago. Names were added because they got liquid. Names were dropped because they did not. By testing today's list over the past, you have selected for things that went on to do well enough to still be in the index, and excluded everything that fell out.

THREE YEARS AGO TODAY still listed still listed still listed dropped out a backtest run on today's list never sees the dashed row at all in the test in the test in the test absent
Survivorship, drawn. The strategy never had the chance to lose money on the name that fell out of the index, because the test could not see it. Your results are quietly measured on winners only.

There is an honest version of this. If your rules would genuinely only ever have traded what was in the index at that time, you need a point-in-time universe. If you accept that you are testing "today's liquid names, historically", say so out loud in your own notes and read the result accordingly. What you must not do is forget which one you did.

So what is a backtest good for?

Having spent a chapter on what it cannot do, here is the honest case for doing one anyway.

It kills bad ideas cheaply. Most strategy ideas are wrong. A test that takes four minutes and shows the idea never worked has just saved you three months of small, demoralising losses. This is the highest-value use and it costs almost nothing.

It finds the shape of a strategy. How often does it trade? How long does it hold? Does it make its money in a few big moves or many small ones? Does it fall apart in a sideways month? None of these are promises about the future, but all of them tell you whether the strategy is something you can actually live with. A strategy that makes money in a way you cannot tolerate is one you will abandon at the worst moment.

It makes assumptions explicit. The act of writing a rule precisely enough for a computer to follow forces you to answer questions you were previously answering by feel. Half the value of a backtest is extracted before you run it.

What it is not is evidence about the future, a reason to size up, or a substitute for forward testing. Treat it as a filter, not a forecast.

What to carry out of this chapter

  • A backtest answers one narrow question: what these exact rules would have done on this exact data. Everything else is inference.
  • Three gaps are yours to close: costs, slippage and data. The fourth is you, and only a journal measures that one.
  • Gross is a number, net is the trade. Costs are a fixed tax on activity, so halving your edge cuts your result by much more than half.
  • Look-ahead does not degrade gracefully. It can make a worthless strategy look excellent.
  • Today's universe tested over the past is a universe of survivors. Know which test you ran.
  • Use a backtest to kill bad ideas and learn a strategy's shape. Do not use it as a forecast.