Why backtest a trading strategy before using it? Here are quick answers to the question:
- To determine whether a theory or hypothetical construct is valid in historical testing; and
- To summarize the overall hypothetical performance of a system and to analyze its various aspects in order to isolate its strong and weak points.
The purpose of testing a pattern or a trading system is simply to find out what will work best on the basis of what had worked best in the past.
You test drive a car before buying it; there is no reason why you shouldn’t test your trading strategy before applying it.
Backtesting is the process of evaluating a trading theory, or model by applying it to historical data. A thorough back test of a trading system should include the following information:
Number of years analysed: Although it is desirable to test as much data as possible, the minimum should be 10 years of recent historical data. Often people test the previous bull market’s data if they figure that the current market resembles a bull market, and vice versa. The most important data tranche is the most recent one as that is what the current market phase is, as you want your trades to work there.
Number of trades analysed: More important than the number of years analysed is the number of trades analysed. A typical pattern should generate at least 20 to 25 trades over the test period in order to support the statistical significance of back-testing results. It would be wrong to assume that a pattern that had formed only a couple of times in the past is a guide or reference to a good trading opportunity in the future.
You may perhaps have come across the term called accuracy while reading statistics. Accuracy usually increases as the number of samples becomes larger and the measurement of deviation or error becomes proportionately smaller. Accuracy is calculated as follows:
Accuracy = (1- (1/ Square root of the sample size))*100.
This concept can be extended to the number of trades analysed. For example, with a sample of four trades, the error is 50%. If a system has had only 4 trades, whether profitable or loss-making, it is very difficult to draw any conclusions about performance expectations. To reduce the error to 10%, the sample size has to be 100 trades. But this could be tricky in respect of a system that might generate only 3 or 4 trades in a year. To compensate for this, the identical pattern can be applied to other markets and the sample size thus increased. By keeping the sample error to no more than 20%, the risk of small sample size can be minimized.
Percentage winning trades: This is not as important as one might think. In reality, few patterns have more than 70 percent winning trades. Patterns that are correct as little as 35 percent of the time can still be good systems, whereas systems that are accurate as much as 90 percent of the time may be bad systems.
Average profit per trade: This measure tells you what the average profit per trade for all the trades has been, minus commission and slippage. The average profit per trade figure is important as it considers all profits and all losses. Some people might question — and legitimately, too — whether, say, a 40-point average profit would vary to a great degree from the underlying Nifty value. For example, a 40-point gain translates to less than 1 percentage gain when the Nifty is trading above 4,000 levels, as opposed to a 2 percentage gain when the Nifty is trading below 2,000 levels. So, it is important to view the trade details in percentage terms as well.
Median profit per trade: In probability theory and statistics, median is described as the numerical value separating the higher half of a sample, a population or a probability distribution, from the lower half. The median of a finite list of numbers can be found by arranging all the observations from lowest value to highest value and picking the middle one. If there is an even number of observations, then there is no single middle value; the median is then usually defined to be the mean of the two middle values.
The median can be used as a measure of location when a distribution is skewed, so it is important to view the median profit per trade (and profit percentage per trade as well) to be in favour of trading strategies. For example if the average profit per trade is, let’s say 0.5% and median profit per trade is -0.2%, it is best to avoid the system.
Largest single losing trade: This measure indicates how much of the drawdown is the result of a single losing trade. In real-life trading, this helps you adjust the initial stop loss. For example, if the average losing trade was Rs. 1,000 and the largest single losing trade was Rs. 8,000, as you would readily guess, a good portion of the average losing trade is borne by the largest losing trade. If you had a better way of managing the largest loser, your overall system performance would be considerably better. You should investigate further the cause for the larger losing trades. In real life trading be prepared to encounter an even higher largest loss, than thrown up by back tested results and brace yourself to handle such situation.
Largest single winning trade: This is more important than the largest single losing trade. Why? Suppose, for example, your total hypothetical profit was Rs. 50,000 for 50 trades, and Rs. 32,250 of this is attributed to only one trade (e.g. long at the close of 15th May 2009), then what you have is a distorted average trade figure. It’s often a good idea to remove such an exceptional single trade from the overall results and re-compute the system performance in order to confirm whether the trading system is actually good enough to trade. In real life trading, be as realistic as possible and prepared that you may never encounter that largest winning trade derived from the back tested results.
Profit factor: Profit factor is the system’s gross profit divided by gross loss. Look for systems that have a profit factor of 2, or higher.
Outlier adjusted profit factor: With any trading pattern, you are going to have one or two exceptional wins. The chances of these trades recurring in the future are very slim and should not be considered in the overall performance summary. It is often a good idea to remove the largest single winning trade while calculating the outlier adjusted profit factor. You might want to consider removing even the top 5% winners. For example, if the number of trades is 40, then remove top 2 largest winners, if number of trades is 60 then remove top 3 largest winners. Look for trading systems with an outlier adjusted profit factor of more than 2.
Maximum drawdown: This is one of the most important aspects of a trading system. A very large drawdown is a negative factor. Maximum drawdown is the largest peak-to-valley loss of a trading system’s historical profit curve. Maximum drawdown can be presented in absolute rupee terms.
Maximum number of consecutive losers / winners: The maximum number of consecutive winning and losing generated is, more often than not, purely psychological. Even using an excellent trading pattern is no guarantee that you will only have winning trades in succession all the time. In other words, there are bound to be a string of consecutive losing trades. But not many traders have the ability to maintain their discipline through four or more successive losing trading trades. Even at the third consecutive loss, you would find many traders ready to abandon their system. To be a winner one would need to weather such storms and be able to take ten or more consecutive losses in one’s stride.
Maximum drawdown (%): As discussed earlier, maximum drawdown is the largest peak-to-valley loss — in absolute rupee terms — of the trading system’s historical profit. Now, suppose you would like to determine the efficiency of a trading strategy in terms of the overall returns it provided on your starting capital. In that case, we can calculate the maximum drawdown as a percentage of the starting capital.
Recovery factor: Because maximum drawdown is measured by the actual amount, it makes little sense to compare it to a buy and hold strategy drawdown during the test period. However, we can compare the recovery factor, which is equal to the absolute value of net profit divided by maximum drawdown. It is desirable for a system to have a recovery factor greater than buy and hold.
Payoff ratio: Payoff ratio is the system’s average profit in rupee terms per winning trade, divided by the average loss in rupee terms per losing trade. Unless the system has a particularly high win/loss ratio, we look for high payoff ratios.
Account size required: The account size required is calculated by considering the capital required at the highest value achieved in the overall testing period, plus the maximum drawdown. The Max Strategy Drawdown value forms the Account Size Required, i.e. the amount of money you must have on the account to start trading the strategy. Only after analyzing Max Strategy Drawdown and determining the Account Size Required we can make an adequate evaluation of the Total Net Profit.
Luck Factor This measure shows how the largest trade compared with the average trade and is calculated by dividing the percentage profit of the largest winning trade by the average percentage profit of all winning trades. The larger the value of the luck factor, the greater portion of the system’s results can be attributed to the largest winning trade, or, luck.
Percent wining months or years: Depending on the time horizon, a trading pattern that averages only one winning month out of twelve, or two winning years out of ten years, is unattractive. You need to look for patterns with at least five profitable months in a year and five profitable years in a ten-year period.
Kora is the author of the recently released book High Profit Trading Patterns published by Vision Books, and is currently co-founder of a quantitative trading portal (http://stocksiq.in) for analysing and backtesting of listed stocks on the Indian Stock Market.