search

AI4Finance-Foundation / FinRL-Tutorials

Tutorials. Please star.
https://ai4finance.org
MIT License
848 stars 349 forks source link

How Stock_NeurIPS2018/agent_a2c.zip was obtained? #43

Open harrytran opened 1 year ago

harrytran commented 1 year ago

Hello,

There is a pre-trained agent (agent_a2c.zip) that is included in Stock_NeurIPS2018. I also reran the notebooks 1 and 2 to train agent_a2c. However the backtest result of the new agent is much worse than the result of the pre-trained agent, even worse than the baseline DJI. Any ideas?

Thanks!

Tagging @ZiyiXia as I see the sample agent was uploaded by you. Was this generated by following the steps in the notebooks or is there anything special about it? It seems to perform much better than the one I re-generated.

ZiyiXia commented 1 year ago

The file agent_a2c.zip is the saved agent after we trained using StableBaseline3 algorithm. Nothing inside the zip file should be changed (actually no need to unzip it at all). To reproduce the performance, I would suggest you try to modify the hyperparameters. For A2C, I think the 50k timesteps is not quite enough. Also, RL's instability is indeed a problem. You might want to run for several times.

harrytran commented 1 year ago

Thanks @ZiyiXia for the suggestion on tuning hyperparameters. Beside that, can you confirm the time period at which your A2C agent was trained?