search

AI4Finance-Foundation / FinRL

FinRL: Financial Reinforcement Learning. 🔥
https://ai4finance.org
MIT License
9.36k stars 2.27k forks source link

SAC model on testing set is allocating same weights to a given TIC on consecutive days #1250

Open arun-dezerv opened 3 days ago

arun-dezerv commented 3 days ago

SAC model on testing set is allocating same weights to a given TIC on consecutive days for nearly 7 years. The model is behaving like an extreme Buy and Hold. I have tried using Optuna to fine tune etc. But still the same result. Below are my hyperparameters

SAC_PARAMS = { "batch_size": 64, "buffer_size": 100000, "learning_rate": 0.001, "learning_starts": 100, "ent_coef": "auto_0.1", }

timesteps = 150000

During training, the sharpe ratio across episodes but the model does not seem to converge. It is possible that my feature space is not very helpful. I am trying to build a long time frame (30 day lookahead) model.

My features include technical features and some macro features.

BruceYanghy commented 2 days ago

Thank you for bringing up the issue. Currently, the FinRL library is extremely poorly maintained. Rest assured, I will reorganize a team to ensure its proper maintenance.

Best regards,

Bruce Yang