Musings with Analyst | Feb 2023

Understanding Backtesting – The Windmill Capital Way

I’m pretty sure you must have come across the term backtesting at some point in your investing journey. However, have you really been able to wrap your head around the concept? If not, read on.

Backtesting is a process of testing your investment strategy historically to see whether the strategy has performed well. In other words, it is a process of looking in the rear-view mirror and gauging the outcome. Please bear in mind that backtesting is just not done to check for returns, but also to assess risk and drawdowns.

Now, there are certain aspects which one needs to be mindful of while backtesting a given strategy, in order to arrive at an accurate conclusion. The first step is to build the stocks universe for the investment strategy retrospectively. This means if you consider the Nifty 500 index as your stocks universe, you’re required to keep refreshing the universe as and when the Nifty 500 index is rebalanced. Allow me to explain this with an example.

Let’s say, on January 1st ‘23, you are backtesting a strategy over the last 5 years. Your portfolio has 15 stocks and you rebalance the portfolio, semi-annually. Essentially, from 2018, you would be required to create a portfolio (from a unique stocks universe) and keep retrospectively rebalancing, on the basis of your strategy.

As you would have guessed already, there are certain biases that come into play while backtesting, and it’s important for the portfolio manager to address them – 

• Look-ahead bias: This bias relates to ‘could-have’ scenarios, wherein data was not available during the time of running the strategy. In the aforementioned example, let’s assume your strategy depends on quarterly results. Now, if you rebalance the portfolio in July, however July’s numbers come in August, the backtesting gets flawed. Looking backwards, you have access to those results, however during that time you wouldn’t have those numbers available.

• Survivorship bias: As the name suggests, this bias relates to the tendency to view the performance of the strategy using the current set of stocks. For instance, if you’re backtesting the strategy from 2018, you cannot be certain that every stock in the current portfolio would have qualified to get included, as company fundamentals change with time. Secondly, one must also account for companies that might have gotten merged, delisted, or would have gone bust for a variety of reasons. There might be a scenario where a company in your historic portfolio went bust and that would affect your backtesting results. Therefore, it is essential to get rid of survivorship bias.

• Time-period bias: This is an extremely common one and relates to a scenario wherein a portfolio manager backtests her strategy over a given time-period, which is best suited for that kind of strategy to show outperformance. For instance, an investment strategy entails picking up stocks that do extremely well in a high inflationary environment. In this case, if the portfolio manager handpicks to backtest her strategy from the start of 2021, that would result in time-period bias, because the strategy coincides with a high inflationary situation.   

You see, backtesting is not a foolproof way of determining the future prospects of an investment strategy. However, it is considered to be a good practice to take into account past performance. Because as Mark Twain famously remarked, ‘History doesn’t repeat itself, but it often rhymes.’

Disclaimer: Investment in securities market are subject to market risks. Read all the related documents carefully before investing. The content in these posts/articles is for informational and educational purposes only and should not be construed as professional financial advice and nor to be construed as an offer to buy /sell or the solicitation of an offer to buy / sell any security or financial products.
Users must make their own investment decisions based on their specific investment objective and financial position and using such independent advisors as they believe necessary.

Windmill Capital Team

Windmill Capital Private Limited is a SEBI registered research analyst (Regn. No. INH200007645) based in Bengaluru at No 51 Le Parc Richmonde, Richmond Road, Shanthala Nagar, Bangalore, Karnataka – 560025 creating Thematic & Quantamental curated stock/ETF portfolios. Data analysis is the heart and soul behind our portfolio construction & with 50+ offerings, we have something for everyone. For more information and disclosures, visit our disclosures page here –https://windmillcapital.smallcase.com/#disclosures