ZhengyaoJiang / PGPortfolio

PGPortfolio: Policy Gradient Portfolio, the source code of "A Deep Reinforcement Learning Framework for the Financial Portfolio Management Problem"(https://arxiv.org/pdf/1706.10059.pdf).
GNU General Public License v3.0
1.75k stars 752 forks source link

Always a portfolio value of approx. 3 #75

Open leokster opened 6 years ago

leokster commented 6 years ago

Hi

I try to optimize my hyperparameter and tried the learning_rate, batch_size and window_size with different values. I always get a return of about 3, which is not very high - no matter what parameters I use. Does anyone have any idea how I could improve performance?

Thank you!

{ "agent_type": "NNAgent", "input": { "coin_number": 16, "coins": [ "ETH", "XRP", "BCH", "LTC", "STR", "XMR", "DASH", "ETC", "XEM", "NXT", "EMC2", "LSK", "ZEC", "DGB", "BTS", "STRAT" ], "end_date": "2018/01/27", "fake_ratio": 1, "feature_number": 3, "global_period": 1800, "is_permed": false, "market": "poloniex", "norm_method": "absolute", "online": false, "portion_reversed": false, "save_memory_mode": false, "start_date": "2015/05/01", "test_portion": 0.08, "volume_average_days": 30, "window_size": 30 }, "layers": [ { "activation_function": "relu", "filter_number": 2, "filter_shape": [ 1, 3 ], "padding": "valid", "regularizer": null, "strides": [ 1, 1 ], "type": "ConvLayer", "weight_decay": 0.0 }, { "activation_function": "relu", "filter_number": 10, "regularizer": "L2", "type": "EIIE_Dense", "weight_decay": 5e-09 }, { "regularizer": "L2", "type": "EIIE_Output_WithW", "weight_decay": 5e-08 } ], "random_seed": 0, "trading": { "buffer_biased": 5e-05, "learning_rate": 0.00028, "rolling_training_steps": 85, "trading_consumption": 0.0025 }, "training": { "batch_size": 160, "buffer_biased": 5e-05, "decay_rate": 1.0, "decay_steps": 50000, "fast_train": true, "learning_rate": 0.00028, "loss_function": "loss_function6", "snap_shot": false, "steps": 80000, "training_method": "GradientDescent" } }

leokster commented 6 years ago

Besides, it is also strange that after 0 training steps I would have this portfolio value in the test set.

`

step 0


the portfolio value on test set is 3.50684

log_mean is 0.000328718

loss_value is -0.000329

log mean without commission fee is 0.000350

==============================

`

dexhunter commented 6 years ago

Hi! What is the backtest result?

leokster commented 6 years ago

Isn't the second part of the "training" the backtest? This is where I get always something around 3. My omegas look also really strange like this [0.0552289 0.05902788 0.05904754 0.05922116 0.05900501 0.05864193 0.05900982 0.05901346 0.05898688 0.05892181 0.05891616 0.05916307 0.05923062 0.05884518 0.05931004 0.05926334 0.05916723]

I attached my train_summary and the net_config from [5]

Thank you!

train_summary.csv.zip

net_config.json.zip