search

AI4Finance-Foundation / FinRL

FinRL: Financial Reinforcement Learning. 🔥
https://ai4finance.org
MIT License
10.14k stars 2.44k forks source link

Can't reproduce test results with FinRL after restarting Jupyter Notebook Kernel #1264

Open Bottagandalf opened 3 months ago

Bottagandalf commented 3 months ago

Hi everyone!

Ive been doing some research for my masters thesis on reinforcement learning using FinRL. I'm mainly using the Copy_of_FinRL_Ensemble_StockTrading_ICAIF_2020.ipynb file and tweaking the parameters slightly. Unfortunately, I'm having issues getting reproducible results. I already set Seeds and my results are reproducible until I restart the Jupyter Notebook Kernel. From there, everything is completely mixed up again.

I looked this issue up on the web and came accross Hashseeds. I tried different methods implementing them but none of it changed the outcome. At this point I think the issue must be somewhere else, but I have no idea what to look out for anymore.

Thanks in advance for your help!

zhumingpassional commented 3 months ago

stability in RL is an important issue, which is not well solved now.

Bottagandalf commented 3 months ago

stability in RL is an important issue, which is not well solved now.

Yeah, that's something I already came across 😢
I was just thinking as I could narrow it down to the Kernel restart, maybe theres something I can do that I didnt think about yet. Would be cool to be able to compare all my results to each other. If thats not possible, I might just do smaller chunks of tests for each parameter only compare them

arun-dezerv commented 3 months ago

I have tried this multiple times and I have not faced issues with the reproducibility of the test set. Why does seed matter for the testing set ? It matters only for model training I think. Please let me know if I am wrong