The performance was measured statistically and financially via a trading simulation taking into account the impact of transaction costs on models with higher trading frequencies. The logic behind the trading simulation is, if profit from a trading simulation is compared solely on the basis of statistical measures, the optimum model from a financial perspective would rarely be chosen.
The NNR model was benchmarked against more traditional regression-based and other benchmark forecasting techniques to determine any added value to the forecasting process. Having constructed a synthetic EUR/USD series for the period up to 4 January 1999, the models were developed using the same in-sample data, 17 October 1994 to 18 May 2000, leaving the remaining period, 19 May 2000 to 3 July 2001, for out-of-sample forecasting.
Forecasting techniques rely on the weaknesses of the efficient market hypothesis, acknowledging the existence of market inefficiencies, with markets displaying even weak signs of predictability. However, FX markets are relatively efficient, reducing the scope of a profitable strategy. Consequently, the FX managed futures industry average Sharpe ratio is only 0.8, although a percentage of winning trades greater than 60% is often required to run a profitable FX trading desk (Grabbe, 1996 as cited in Bellgard and Goldschmidt, 1999: 10). In this respect, it is worth noting that only one of our models reached a 60% winning trades accuracy, namely the MACD model at 60.00%. Nevertheless, all of the models examined in this chapter achieved an out-of-sample Sharpe ratio higher than 0.8, the highest of which was again the NNR model at 2.57. This seems to confirm that the use of quantitative trading is more appropriate in a fund management than in a treasury type of context.
Forecasting techniques are dependent on the quality and nature of the data used. If the solution to a problem is not within the data, then no technique can extract it. In addition, sufficient information should be contained within the in-sample period to be representative of all cases within the out-of-sample period. For example, a downward trending series typically has more falls represented in the data than rises. The EUR/USD is such a series within the in-sample period. Consequently, the forecasting techniques used are estimated using more negative values than positive values. The probable implication is that the models are more likely to successfully forecast a fall in the EUR/USD, as indicated by our results, with all models forecasting a higher percentage of winning down periods than winning up periods. However, the na¨ıve model does not learn to generalise per se, and as a result has the smallest difference between the number of winning up to winning down periods.
Overall our results confirm the credibility and potential of regression models and particularlyNNR models as a forecasting technique. However, while NNR models offer a promising alternative to more traditional techniques, they suffer from a number of limitations.
They are not the panacea. One of the major disadvantages is the inability to explain their reasoning, which has led some to consider that “neural nets are truly black boxes. Once you have trained a neural net and are generating predictions, you still do not know why the decisions are being made and can’t find out by just looking at the net. It is not unlike attempting to capture the structure of knowledge by dissecting the human brain” (Fishman et al., 1991 as cited in El-Shazly and El-Shazly, 1997: 355). In essence, the neural network learning procedure is not very transparent, requiring a lot of understanding. In addition, statistical inference techniques such as significance testing cannot always be applied, resulting in a reliance on a heuristic approach. The complexity of NNR models suggests that they are capable of superior forecasts, as shown in this chapter, however this is not always the case. They are essentially nonlinear techniques and may be less capable in linear applications than traditional forecasting techniques (Balkin and Ord, 2000; Campbell et al., 1997; Lisboa and Vellido, 2000; Refenes and Zaidi, 1993).
Although the results support the success of neural network models in financial applications, there is room for increased success. Such a possibility lies with optimising the neural network model on a financial criterion, and not a mathematical criterion. As the profitability of a trading strategy relies on correctly forecasting the direction of change, namely CDC, to optimise the neural network model on such a measure could improve trading performance. However, backpropagation networks optimise by minimising a differentiable function such as squared error, they cannot minimise a function based on loss, or conversely, maximise a function based on profit. Notwithstanding, there is possibility to explore this idea further, provided the neural network software has the ability to select such an optimisation criterion.
Future work might also include the addition of hourly data as a possible explanatory variable. Alternatively, the use of first differences instead of rates of return series may be investigated, as first differences are perhaps the most effective way to generate data sets for neural network learning (Mehta, 1995).
Further investigation into RNN models is possible, or into combining forecasts. Many researchers agree that individual forecasting methods are misspecified in some manner, suggesting that combining multiple forecasts leads to increased forecast accuracy (Dunis and Huang, 2002). However, initial investigations proved unsuccessful, with the NNR model remaining the “best” model. Two simple model combinations were examined, a simple averaging of the na¨ıve, ARMA and NNR model forecasts, and a regressiontype combined forecast using the na¨ıve, ARMA and NNR models.
The NNR model was benchmarked against more traditional regression-based and other benchmark forecasting techniques to determine any added value to the forecasting process. Having constructed a synthetic EUR/USD series for the period up to 4 January 1999, the models were developed using the same in-sample data, 17 October 1994 to 18 May 2000, leaving the remaining period, 19 May 2000 to 3 July 2001, for out-of-sample forecasting.
Forecasting techniques rely on the weaknesses of the efficient market hypothesis, acknowledging the existence of market inefficiencies, with markets displaying even weak signs of predictability. However, FX markets are relatively efficient, reducing the scope of a profitable strategy. Consequently, the FX managed futures industry average Sharpe ratio is only 0.8, although a percentage of winning trades greater than 60% is often required to run a profitable FX trading desk (Grabbe, 1996 as cited in Bellgard and Goldschmidt, 1999: 10). In this respect, it is worth noting that only one of our models reached a 60% winning trades accuracy, namely the MACD model at 60.00%. Nevertheless, all of the models examined in this chapter achieved an out-of-sample Sharpe ratio higher than 0.8, the highest of which was again the NNR model at 2.57. This seems to confirm that the use of quantitative trading is more appropriate in a fund management than in a treasury type of context.
Forecasting techniques are dependent on the quality and nature of the data used. If the solution to a problem is not within the data, then no technique can extract it. In addition, sufficient information should be contained within the in-sample period to be representative of all cases within the out-of-sample period. For example, a downward trending series typically has more falls represented in the data than rises. The EUR/USD is such a series within the in-sample period. Consequently, the forecasting techniques used are estimated using more negative values than positive values. The probable implication is that the models are more likely to successfully forecast a fall in the EUR/USD, as indicated by our results, with all models forecasting a higher percentage of winning down periods than winning up periods. However, the na¨ıve model does not learn to generalise per se, and as a result has the smallest difference between the number of winning up to winning down periods.
Overall our results confirm the credibility and potential of regression models and particularlyNNR models as a forecasting technique. However, while NNR models offer a promising alternative to more traditional techniques, they suffer from a number of limitations.
They are not the panacea. One of the major disadvantages is the inability to explain their reasoning, which has led some to consider that “neural nets are truly black boxes. Once you have trained a neural net and are generating predictions, you still do not know why the decisions are being made and can’t find out by just looking at the net. It is not unlike attempting to capture the structure of knowledge by dissecting the human brain” (Fishman et al., 1991 as cited in El-Shazly and El-Shazly, 1997: 355). In essence, the neural network learning procedure is not very transparent, requiring a lot of understanding. In addition, statistical inference techniques such as significance testing cannot always be applied, resulting in a reliance on a heuristic approach. The complexity of NNR models suggests that they are capable of superior forecasts, as shown in this chapter, however this is not always the case. They are essentially nonlinear techniques and may be less capable in linear applications than traditional forecasting techniques (Balkin and Ord, 2000; Campbell et al., 1997; Lisboa and Vellido, 2000; Refenes and Zaidi, 1993).
Although the results support the success of neural network models in financial applications, there is room for increased success. Such a possibility lies with optimising the neural network model on a financial criterion, and not a mathematical criterion. As the profitability of a trading strategy relies on correctly forecasting the direction of change, namely CDC, to optimise the neural network model on such a measure could improve trading performance. However, backpropagation networks optimise by minimising a differentiable function such as squared error, they cannot minimise a function based on loss, or conversely, maximise a function based on profit. Notwithstanding, there is possibility to explore this idea further, provided the neural network software has the ability to select such an optimisation criterion.
Future work might also include the addition of hourly data as a possible explanatory variable. Alternatively, the use of first differences instead of rates of return series may be investigated, as first differences are perhaps the most effective way to generate data sets for neural network learning (Mehta, 1995).
Further investigation into RNN models is possible, or into combining forecasts. Many researchers agree that individual forecasting methods are misspecified in some manner, suggesting that combining multiple forecasts leads to increased forecast accuracy (Dunis and Huang, 2002). However, initial investigations proved unsuccessful, with the NNR model remaining the “best” model. Two simple model combinations were examined, a simple averaging of the na¨ıve, ARMA and NNR model forecasts, and a regressiontype combined forecast using the na¨ıve, ARMA and NNR models.
No comments:
Post a Comment