search

AI4Finance-Foundation / FinRL

FinRL: Financial Reinforcement Learning. 🔥
https://ai4finance.org
MIT License
9.36k stars 2.27k forks source link

Graph Neural Network Architecture for Portfolio Optimization #1206

Closed C4i0kun closed 2 months ago

C4i0kun commented 2 months ago

Hello!

This pull request adds a new architecture that can be used in the portfolio optimization task. The GPM architecture adds relational graph convolutional neural networks to the EI3 architecture (that is already included in FinRL).

To be able to work with this new architecture, new features where added to PortfolioOptimizationEnv to allow the state space to be bigger than the portfolio so that the agent can make use of informations about stocks that are not included in the portfolio to take action. These modifications will not change the behavior of already implemented code.

Additionally, an example using GPM was added to the examples folder (FinRL_GPM_Demo.ipynb) and it makes use of a well-known database with temporal and relational data that I modified so that it can to be used with FinRL easily, you can find the modified database here.

zhumingpassional commented 2 months ago

@C4i0kun great! very good work.

zhumingpassional commented 2 months ago

have you tested it? does it work well now?

C4i0kun commented 2 months ago

have you tested it? does it work well now?

Yes, the previous example using simple convolutional architectures works just like before and the new architecture with graph neural networks is also learning an optimal policy. In the provided example, I only train 2 episodes because this new architecture is very "computational heavy", since we are using a graph that represents the whole NASDAQ market.

I noticed that this new architecture is learning a lot faster than the previous ones: even though training takes a lot more time, the performance increase beween two consecutive episodes is considerably bigger. I'm still doing hyperparameter tuning, so I haven't been able to achieve the results in the original article, but I'm very optimistic!

zhumingpassional commented 2 months ago

good. I encourage you to wok towards GNN based methods and submit new PRs. you said "training takes a lot more time“, did you use GPU when training?

zhumingpassional commented 2 months ago

image I think the data should be moved to self.device. If we are using GPU, it will be better. do you think so?