search

joshuaulrich / TTR

Technical analysis and other functions to construct technical trading rules with R
GNU General Public License v2.0
330 stars 103 forks source link

[R-Forge #2043] DPO implementation looks into the future #1

Closed joshuaulrich closed 9 years ago

joshuaulrich commented 9 years ago

Submitted by: Mirko Fit Assigned to: Joshua Ulrich R-Forge link

The implementation of the detrended price oscillator looks into the future by shifting moving averages backwards.

Why this is problematic:

1.) The current values of DPO contain information about the future -> I used genetic programming to find trading rules in forex markets and found a lot of unrealisticly successful trading rules with that approach. By now it looks like the future information within DPO made it easy to find trading rules with high forecasting abilities.

2.) It is impossible to use it in an event based system, as the information of future MA values is not available at calculation time

From my point of view, every indicator that produces n/a values at the end of the time series, may contain future information and is therefore not practically usable.

In case of the DPO, a hotfix could be to shift the final result forward by 'shift' (option). The most clean I can imagine is to implement it via some buffer (like shift registers in hardware). One would create a buffer of size 'shift' and store the past prices in there. When the new value of DPO is calculated, one can substract the MA value from the price of 'shift' periods ago.

Followups:

Date: 2013-02-24 23:59 Sender: Joshua Ulrich This isn't a bug; it's how the indicator is calculated. See the references below: http://www.fmlabs.com/reference/DPO.htm http://www.equis.com/Customer/Resources/TAAZ/?c=3&p=48 http://stockcharts.com/school/doku.php?id=chart_school:technical_indicators:detrended_price_osci

systats commented 3 years ago

Hi Joshua,

i wonder which value it has to incorporate it into a model as it is leaking the future? Its seems also to be the only obviously leaking feature. Maybe renaming it as looking_future_DPO prevents use in the first place. Everything else works fine for me so far. Thank you!

joshuaulrich commented 3 years ago

Hi @systats, I don't think I should have included DPO() in TTR for that reason, but I think it's too late to remove it or change the name now.

Maybe I could add a message about it looking into the future the first time it's called... with a global option to turn it off by default... but that could also be annoying too. I might be able to find a way to have the message thrown once per TTR install, with the same global option to disable it completely.

Thoughts?

braverock commented 3 years ago

It seems to me that the warning in the documentation about look-ahead-bias is sufficient.

Plenty of published papers make the mistake of demeaning the entire series, for example. It makes the inference suspect, certainly, but it is just a 'detrending' methodology.

Likewise, the DPO implementation's use of a smoother (a two-sided filter) rather than a one-sided backwards looking filter is just the way it is specified.

Technical analysis models are more useful as starting points for developing something modern anyway than they are as models on their own...