ZhengyaoJiang / PGPortfolio

PGPortfolio: Policy Gradient Portfolio, the source code of "A Deep Reinforcement Learning Framework for the Financial Portfolio Management Problem"(https://arxiv.org/pdf/1706.10059.pdf).
GNU General Public License v3.0
1.73k stars 748 forks source link

back testing to aug 2018, the model -50% and also lose to legacy model. #103

Open JiniusDnn opened 6 years ago

JiniusDnn commented 6 years ago

Author only use the up trend data during crypto history to verify the model. when down trend comes, model's performance is terrible.

The weights from down trend, RL training suppose change all alt coin's weights to near 0 as a defence strategy , but this model's RL part never demonstrate this type of wisdom.

JiniusDnn commented 6 years ago

image

ErnstDinkelmann commented 6 years ago

@JiniusDnn Did you optimize the hyper-parameters again? There has been a few discussion on the performance recently. The answer is generally to re-optimize the hyper-parameters (ie some of things in the net_config.json file). I'm looking to test that theory, as I've seen the same as you above. The code to do this has not been included, so you need to do this yourself. I think that's part of their IP is how to do this efficiently.

What I do wonder though is whether the structure of the NN might not be able to work in all market circumstances. I mean it's a very specific design. And that's not something you can readily loop over. It requires some logic first. I'm not really clear why this structure works intuitively speaking.

There is the matter you allude to, the moving out of assets into "cash", which I think is not working great. This has also been mentioned by other. I think one way to solve this may be to include BTC (the "cash" instrument) as an actual coin as well. In other words to not try to work it in at the last step of the NN with a btc_bias variable, but have it evaluate that as a coin as well. Yes the features will be fixed for the entire window for this coin (BTC), eg price = [1,1,1,1,...,1,1,1], but I think it might better allow the network to just see it as another option. Just an idea - I've not tested.

JiniusDnn commented 6 years ago

@ErnstDinkelmann thanks for your help , I'll try your strategy. 1.) first try optmize the parameter,byside the config file you send to me. 2.) switching btc to usd as cash to reverify the performance. 3.) figure out a solution the NN can be trained to all weights to near 0 in downtrend. will post progress if made.

jshin47 commented 5 years ago

any luck with getting good returns?

GerardBCN commented 5 years ago

Just a comment on this. Parameter optimization seems to play an important role to optimize the returns. An example of such is if one changes the default 30-minute candlesticks for 4-h candlesticks, I got 2x revenue on a model trained from the default initial date until 30/09/2018 with 0.2 of test data (so test data was more or less from February 2018 to end of September 2018).

I found interesting and surprising to see that the default input of 30(min)*50candles=25h was enough information for the model to take a trading decision... My intuition tells me that in non-day trading (and specially when big downtrends are on play) the model gets more robust when increasing the timeframe the model sees to get a decision from (e.g. from 25h to 8.3days when increasing the candlestick from 30min to 4h). pgp_progression_04hcandle

Masonlxy commented 5 years ago

@JiniusDnn

image

@JiniusDnn I meet the same question with you,could give me some advice