In this episode, Walter leans on the topic about the downsides of using backtesting and some of the common mistakes of using this method. He also gives some useful insider tips when backtesting to get the best and accurate data possible.
But, does having this number of computer screens actually contribute a tremendous amount to the success of your trades? Or should the focus be more on your trading psychology and execution?
Download (Duration: 08:16 / 9.46 MB)
In This Episode:
00:41 – limited range
01:32 – limited slice
03:10 – two big mistakes
05:20 – ten years of data
07:34 – snapback system
Tweetables:
Never assume that your system is going to work in all markets. [Click To Tweet].
Test on different types of markets and see what happens. [Click To Tweet].
Get the breadth of different types of market under your belt. [Click To Tweet].
Announcer: Sometimes, forex trading is a wild and wooly place to be. That’s why Hugh is here, to post your questions to Walter, the naked forex guy. Hugh’s got questions and Walter’s got the answers. Here at the Truth About FX Podcast.
Hugh: Hi, Walter. A lot of educators talk about the benefits of backtesting but what are the downsides?
Walter: Wow, what a great question! One of the things I would say about backtesting is if you are not really careful, you can make a couple of mistakes. The first mistake would be that you would backtest a very limited range.
Let’s say for example, you backtested the EUR on the daily chart between say 2002-2004 or something like that or even 2002-2003, what you might come away with is a system that works really, really good in a trending market but does not work so well in a choppy sideways, swing up, swing down market.
That is one thing that I would guard against. The other thing would be, in other words, what you would want to do is you want to make sure that you test in different types of markets.
You want to make sure that you know based on your backtesting “Okay, if there’s a trending market, my system is not going to do so well” or, “If there is a sideways market, wow, my system is going to do really well” or whatever. You have to know that.
If you only take a limited slice of historical charts to use in your backtesting to verify your system and to build your confidence, then you are going to be in trouble later on when the market does not give you exactly what you backtested.
The other mistake would be to assume that your system that you backtested is going to work in all markets. The real point here is if I am backtesting the CAD/JPY and if I’ve got a lot of dials and switches and things that I can fine tune and indicators are famous for this.
Let’s say, I am doing a CAD/JPY system and I adjust the MACD, I adjust the moving average and maybe I’ve got like a Stochastic or Momentum or something and I just fine tuned all those that fits my systems really, really well, I cannot assume then that is going to work if I go ahead and trade the GBP or the EUR/JPY something like that.
The reason why I like simple strategy is not only because it makes it easy to make a decision but the other piece of that is it’s hard to curve fit or over fit your system if it’s so simple that you can’t really dial it in and get really, really tight to the data, if that make any sense.
I like really simple, robust strategies and if you are really honing your strategy so it works really, really well on the data set that you’ve tested, then it often means that when you take that set up, that system and you apply it to a new data set which could be the live markets, you’re in big trouble. Those are two big mistakes I think people make.
They do not test different types of markets to see what happens the personality of their system and the types of markets. The other one would be over-fitting like curve fitting. Over fitting your system to a particular market so that means and, of course, it does not quite apply as well to the other markets. Those are the two things that I would say.
Hugh: I see that a lot especially with the EA’s and the robots, they have a tremendous one year track record. But, unless that was 2004, you are not getting the same result, right?
Walter: Exactly, that is right. You got it. Cool! Great question.
Hugh: Thanks. Do you have a rule of thumb as to how many markets to test or how long to test or anything like that?
Walter: One thing that I would say, I’ll definitely go back at least ten years when I am looking on strategy and then what I’ll do… For example, let say I go Kelsey , if this works on EUR. If you want, I can post underneath this a video so everyone listening to this can get it. I’ll post an ebook on Kelsey strategy that I use which is a very simple price action based strategy that is based on a particular candle type. Now, when I went and look at that strategy, I back tested it and first the EUR, the GBP, the AUD these sorts of things and I always look at ten years then I’ll go back and do another pair and do ten years on that one and so forth and so forth.
I won’t actually trade it live until I backtested it, that particular market, and over the last ten years. I figure, in most cases, that’ll give me a wide range of different market conditions. It may not but that is my assumption.
The other thing is you can eyeball it. You can look at it and say “Has the market been going mostly up?” So, if you look at the AUD for 2015, it was going mostly down. These are the sorts of things you can eyeball but looking at the chart too.
I guess to answer your question, I typically will go with ten years of data and I won’t trade it until I’ve actually backtested it. I don’t just backtest the strategy in the EUR then go “Well, I am going to trade in the NZD, I want to trade in the AUD, I’m going to trade the CAD” and all that.
I definitely want to make sure it works in that particular markets. When I came with the special derivative of the belts set up which I will show you in the shownotes of this podcast, I wanted to make sure that it was going to work in different types of market.
I had to go back and test all of them and that is the way I look at it. I look at ten years and I definitely need to test the market before I go ahead and trade that market. That is how I do it.
Hugh: I see. Cool! What if somebody ask the opposite problem, like maybe they are testing a system on the thirty-minute chart or something and they have? If they want to look at ten years of data and there’s a lot of testing so, what would you recommend in that case?
Walter: Yeah, right. Good question! I assume, I am looking at daily charts, I am looking at daily and weekly charts, things like that and ten years with weekly charts may not be enough but I need to go back a little bit farther.
If you are looking at thirty-minute charts — I’m sorry, I totally forgot it that there are a lot of people that trade the lower timeframes — definitely, I would go in volume. This is what I tell my students, if you don’t have at least 500 trades under your belt then do not even think about going live with it. That would be the minimum sample size, would be 500 trades. Again, the idea here is it’s the number of trades, yes, but more importantly the type of market that it’s trading.
If you’ve got a reversion to the mean type system and you wait for the market to get really far away and then you are basically betting on it snapping back to a mid point like the middle of the Bollinger Band or whatever. If you are doing that, you want to make sure that you take a lot of those trades during, or at least see potential trades trending in the markets, because that Bollinger Bands snap back system is going to work really, really well when you are trading a sideways market.
When you are in a trending market, it is going to be a lot tougher. That is the more important thing to me, is you get the breadth of different types of market under your belt.
Certainly, the number is critical too because the more you take, the more likely you are to hit different types of markets. I would just say “Look, ten years, if you are trading the daily charts, 500 minimum if you are trading anything lower than the four-hour,” basically.
Hugh: Very cool. Awesome! Thanks
Walter: Thanks. See you.
Shownotes:
Belt’s Setup Derivatives Video
Podcast: Play in new window | Download
Subscribe: Apple Podcasts | Android | RSS