EP148: Why Is Backtesting Important?

Truth About Forex

6 years ago

In this episode of Truth About FX, Walter gives light on the importance of backtesting… But what if you don’t have enough data? He talks on why statistics is important and how this can help you see results even if you don’t have enough data.

According to Walter, it all boils down to being confident with your system and how you work your way towards your trading goals

Download (Duration: 07:18 / 8.4 MB)

In This Episode:
00:38 – move forward
02:92 – shovel in the dirt
04:04 – reward yourself
05:39 – absolute speed

Tweetables:
Reward yourself along the way [Click To Tweet].
It’s all about the expectations [Click To Tweet].
Focus on what you have [Click To Tweet].

Announcer: Sometimes, forex trading is a wild and wooly place to be. That’s why Hugh is here, to post your questions to Walter, the naked forex guy. Hugh’s got questions and Walter’s got the answers. Here at the Truth About FX Podcast.

Hugh: Hi, Walter. I’ve got a question for you. What if somebody wants to backtest and understand the importance of backtesting but they just go so slow that it’s frustrating for them because they can’t get enough data. How would you recommend they move forward?

Walter: There’s a couple of suppositions in there. One is they don’t have enough data. I don’t know. Maybe they do, maybe they don’t. We have to look at that and make sure that’s the case. But, let’s say that this trader feels like he needs to have — I said he because most traders are men but could be a woman. Most really good traders are women. So, let’s say she says, “I need to have 500 trades and I’m only able to knock out 10 in an hour and I don’t want to spend 50 hours doing this.”

Maybe that’s what is going on. I don’t know where that number came from. There’s a point of diminishing returns. In statistics, usually what happens is… I mean it depends on what the question is and so the question dictates the statistic that you use but, in most cases, I hear people say things like “You must have 30 in order to have a statistical significance, blah, blah, blah.” That’s a load of hooey. You could have 10 and it can be significant.

Let me put it this way. Let’s say you were out there digging in the forest and you’re looking for gold. You’re digging in the forest and you’re looking for gold. Holy crap, the first time you sink your shovel into the dirt, ting! You hear this sound and I’ll be darn if I didn’t just see a gold nugget with my shovel.

Now, let me ask you something. Would you be more willing to say, “I think there’s gold in that dirt.” Or, would you be more likely to say, “You know what? We really need to dig 30 tons of dirt out of here before we know if there’s any gold in that dirt.”

That’s the same thing with statistics. You can actually have a really small sample and if it’s really a strong result, there’s probably going to be a lot there. That’s the same thing. Like, that’s why I used the gold digging thing. I just came up with it.

The idea there is if you see gold straight away, then there’s probably gold down there. You could’ve been lucky. There’s a chance that maybe you’ve just got lucky and for whatever reason, you dug the only gold nugget from here to Seattle and you’ve got lucky. Maybe that’s true but it’s probably not true. Probably it’s full of gold and that’s probably what’s going on here.

So, when people say things like you need to have X number of trades to be statistically significant blah, blah, blah, they’re usually just pulling that out of the last person they heard it from. Usually, the number people say is 30 or a 100. It makes no sense.

The answer is it depends on what the question is. There’s more and I can give some more details but I don’t want to be get bogged down here. The other part of this question is let’s just say that they’re right that they do need X number of trades.

The real advantage of taking 500 trades is not to have a statistically significant data result or win rate or whatever. It’s more that you’ve built confidence in the system and you have all these system under your belt. That’s really the value. But, anyway, we’ll move beyond that.

We’ll go let’s say that this person is right and she does need 500 trades or whatever the number is. She’s really taking a long time to do that. What you might do then is just make sure that you reward yourself along the way.

You do something like every time you get your tenth trades, do something nice for yourself. Buy yourself a mango, a coffee, or whatever you like to do, or go to the movies and chunk it out.

It’s kind of like the guy who says, “I want to make a Million Dollars this year.” Backwards engineer that. A Million Dollars, how much is that a month? It’s like 87,000 a month or whatever it is and you work it out. You work it out in a day or whatever.

Same thing with the 500 trades. If you want to do 500 trades between now and Christmas, you work out how many days there are. How many days of the week are you actually going to backtest. How many hours can you do it and just go, chunk it out to little bits and reward yourself along the way so you can get to your overall goal. That’s what I would recommend but again I don’t really know what the reasoning is here and why.

It could be the strategy doesn’t have a lot of signals and so, that’s why the person is frustrated because you have to spend a lot of time on Forex Tester and getting few trades in return. That could be the case and that’s just going to mean that you’re going to have to chunk it out into a bunch of sessions and make sure that you reward yourself as you go.

The other reason, of course, could be because this person feels like that they need more trades to get some real data and that may or may not be true. I don’t know if that helps. Does that make sense or do you think there’s something else to that question?

Hugh: Yeah, totally makes sense. I think it is about the expectations and not necessarily the absolute speed or the number of trades.

Walter: Yeah and you’ll see it’s settle down too. Like as you go, you have 20 trades under your belt, then 50, then a 100. What you’ll see, the numbers should start to settle down. In the beginning, you might think, “Wow, this has a 65% win rate after 20 trades.” And then, after 50 trades you go, “It’s going to be 55% win rate.” And then, after a 100 trades, you go, “It’s going to be about 56% win rate.” And then, after 200 trades, you go, “Yeah, it’s about 55.9%.”

You sort of work it out as time goes. It becomes pretty obvious what’s going on there but it takes time to get there. Just reward yourself. I think sometimes we focus on what we don’t have or we’re not when we can just look back and go, “Wow, I’ve done this much already.” That can be really valuable too.

Just knowing that you’ve been able to get that far. As far as you have coming and don’t focus on the hole. Focus on how far you’ve come and reward yourself for that because you’re almost there. I wouldn’t worry about the fact that you still have another 300 trades to go. Look, you’ve already done 200, you’re almost halfway there. That’s great.