search

AI4Finance-Foundation / FinRL

FinRL: Financial Reinforcement Learning. 🔥
https://ai4finance.org
MIT License
9.61k stars 2.33k forks source link

Paper replication questions - portfolio allocation #595

Closed pbezz1 closed 2 years ago

pbezz1 commented 2 years ago

I am interested to replicate the results of the paper with title "FINRL: A Deep Reinforcement Learning Library for Automated Stock Trading in Quantitative Finance" of NeurIPS 2020, particularly the case of portfolio allocation. I am following the tutorial which is very well described in this notebook: https://github.com/AI4Finance-Foundation/FinRL/blob/master/tutorials/1-Introduction/FinRL_PortfolioAllocation_NeurIPS_2020.ipynb, but I have a couple of issues which I am facing when trying to replicate the results of the paper specifically Figure 5 and Table 2. So I would really like to get some insight on how to proceed please:

  1. Did you use any hyperparameter tuning to produce the results of Table 2 and Figure 5?
  2. Have you set turbulence parameter to True?
  3. Have you added technical indicators to the feature list?
  4. Could you provide the parameters you used for the DDPG and TD3 algorithms?

I would greatly appreciate your answers at your earliest convenience so that I can continue with my study.

Thank you in advance.

pbezz1 commented 2 years ago

@XiaoYangLiu-FinRL could anybody help me with these questions please?

Athe-kunal commented 2 years ago

In RL, the training part is sensitive and it depends on the initial seed value and hyperparameters. So I would suggest you do any hyperparameter tuning by taking help from tutorials from here. The turbulence parameters are set to True and technical indicators were used as mentioned in the paper. You can re-run your training for 5 different seed values and showcase the average value. This is the general norm in all the RL papers due to the high sensitivity of RL algorithms. I am not sure of the approach in paper, but yes you can re-run the training and showcase the average results or do Hyperparmaeter optimization to get the best results.

pbezz1 commented 2 years ago

Thanks for the comment. Is it possible to know which hyperparameters were tuned for the paper please? and whether these were tuned using Optuna?