What are the Risks of Over-Optimizing an MT4 Expert Advisor

The power of optimization is that allows you to test hundreds of possible parameter settings in order to find the most profitable. However, as with all powerful things, it can be abused. Like leverage, optimization has the power to enhance a strategy and return great profits, if strictly controlled, or it can destroy a strategy and one’s account if not.

In the search for the best parameters, it is very easy to fit the data to the history and gain the illusion of “huge profits” that works on history only.  This is called overoptimization, and it is so easy to do it is almost unavoidable. Find out on this article what are the risks of over-optimizing an MT4 Expert Advisor (EA) and how to avoid the dangerous deception of over-optimization.

Overoptimization can be dangerous in its self-deception. It can create a false sense of EA profitability that encourages the GREED within us to try our “Utopian” EA on a real account –and thereby subject our account to the flame of what in fact is a truly terrible system that can burn it the ground.

It is foolish to forsake optimization because we will then miss out finding the best parameter setting to a potentially profitable strategy and may instead be stuck with the arbitrary parameter setting that is wrong. However, it is so very easy to cross the line into overoptimization and its treacherous consequences. How do we find the golden mean?

Just because we find something potentially dirty in the process of optimization does not mean we have to throw the baby out with the bathwater.  Optimization is worth pursuing so long as one takes measures to reduce the risks of overoptimization.

There are five steps one can take to minimize overoptimization:

Step #1: Optimize on few parameters as possible.

Do not add unnecessary combinations of rules (indicators, signals, exits and filters). With each additional rule you add to the strategy, you increase the possibility of curve-fitting your strategy to historical data.

Step #2: Sounding Parameters of Optimal ones should also be profitable.

Whenever you optimize a parameter to find a profitable one, make sure that the surrounding parameters are also profitable. If the optimal length of the indicator turns out to be 18, make sure that the surrounding parameters below (15, 16,17) and above ( 19, 20, 21) are also profitable.   If any of the parameters on either side of the optimum is not profitable, this should be a red flag.

Step #3: Optimize over large data sets (history, bars and trades).

It is much easier to curve fit the smaller the history you are optimizing on. It is also much easier to curve fit the larger timeframe you are optimizing on. Generally speaking, the larger the timeframe, the more history you must optimize upon because you need so much more history to get a reliable bar sample size. The smaller the timeframe, the less history you need to work because you have more bars. However, you still need a fair bit of history because history shows different behaviors in the markets.

Step #4: Optimal parameters should work on out of sample (walk forward) data.

Do the optimal parameters work on out of sample data, that is, on data from another, non-overlapping time period?

When you perform tests of your system on historical data, it is always useful to “save” some time period of historical data, which will be used for other testing purposes. Generally, there are two types of data: in-sample data and out-of-sample data. In-sample data are the data, which were initially used for tests and optimization; out-of-sample data are the data, which have been saved. Saving data for further tests is very important for the process of evaluation.

Therefore, you provide yourself with data that have not been used before in the optimization model. So, out-of-sample data will not influence your system and you will be able to find out real performance of your trading system. Simply said, you will see the possible behavior of your system during real-life trading.

Before starting any optimization or backtesting you should save some amount of the historical data, which will be used for out-of-sample testing. For example, you can split the historical data into three parts and set one part aside for out-of-sample testing.

Then you should use only the remaining two parts of the historical data (in-sample data) for the initial testing and optimization. As you can see on the Picture 1, one-third of the historical data on the time line is set aside for out-of-sample testing, and the other two parts are used for the initial testing as the in-sample data.

For example, you can test your system on data from 2000-2008, this will be your control sample history, and then when you find the optimal parameters, you can test them on a “walk forward” history set of 2008-2010. If your optimal parameter performs poorly in the out of sample data, it means it has been over-optimized.

The order of data placement on the line isn’t quite typical: as a rule, the out-of-sample data stands not in the beginning of testing, but right before the forward performance.

As soon as you’ve tested your system on the in-sample data, you can continue the process and apply the system to the out-of-sample data. It is done in order to give you an opportunity to compare the results of system performance on in-sample and out-of-sample data.

Correlation is the process of finding likenesses between the trends and the performances of in-sample and out-of-sample data. Almost all trading platforms allow you to compare reports about performance of your system, created during tests. If the correlation between the data sets is high, then it is quite possible that your strategy will be successful and will show good results during forward performance tests and real-time trade.

If correlation between two data sets is insignificant, like on the left chart, there is high probability that the system won’t work well during real-time trade because of over-optimization. In case of strong correlation (the right chart) you can proceed and perform additional test referred to as forward performance test.

Step #5: Optimal parameters should work on forward performance data.

Forward performance testing is another tool for testing performance of your system. This set of out-of-sample data is often called paper trading, because all operations are executed “virtually”. During forward performance testing a possible situation in a live market is simulated, and the system works according to its parameters and logic as if it traded live. It means that any profit, loss, entries, and exits are recorded, but no real operations are performed.

Remember: it is very important to follow the logic of the system as precisely as possible, or you won’t be able to evaluate forward performance testing results. You should honestly specify all trade entries and exits without any exceptions and without any explains. Only in this case you will be able to evaluate the system properly.

Many brokers have an option of opening a virtual trading account, which you can use for practice: open trades, calculate profit and loss, etc. This account allows you to test you system without threat of losing real money.

In case of good correlation between all data sets your system is ready for using it for a real trade.