AI4Finance-Foundation / FinRL

FinRL: Financial Reinforcement Learning. 🔥
https://ai4finance.org
MIT License
10.03k stars 2.41k forks source link

Received ValueError while running China_A_share_market_tushare.ipynb. #1110

Open godblessbb opened 1 year ago

godblessbb commented 1 year ago

I'm trying to move China_A_share_market_tushare.ipynb to run under the PyCharm environment, and replaced the data obtained from the tushare API with my own data. I encountered an error when running this line of code:

env_train, _ = e_train_gym.get_sb_env()

The error message is as follows:

Traceback (most recent call last):
  File "/Users/bicaihua/PycharmProjects/Quant/Gym.py", line 99, in <module>
    env_train, _ = e_train_gym.get_sb_env()
  File "/Users/bicaihua/PycharmProjects/Quant/FinRL/finrl/meta/env_stock_trading/env_stocktrading.py", line 563, in get_sb_env
    obs = e.reset()
  File "/Users/bicaihua/PycharmProjects/Quant/venv/lib/python3.8/site-packages/stable_baselines3/common/vec_env/dummy_vec_env.py", line 75, in reset
    self._save_obs(env_idx, obs)
  File "/Users/bicaihua/PycharmProjects/Quant/venv/lib/python3.8/site-packages/stable_baselines3/common/vec_env/dummy_vec_env.py", line 105, in _save_obs
    self.buf_obs[key][env_idx] = obs
ValueError: setting an array element with a sequence. The requested array would exceed the maximum number of dimension of 1.

It seems that there is a problem when the train is encapsulated into the DummyVecEnv class and then the reset operation is performed. But I printed the shape of the environment at every step, and it seems to be normal. Has anyone encountered a similar problem?

godblessbb commented 1 year ago

I took care of the error on my own. Performing a reset on the environment returns self.state and an empty dictionary. I solved the problem by deleting the empty dictionary. But it is not clear if this will cause further problems. We'll see what happens.

Also, the sample files for this project are full of all sorts of errors. Questions are not answered as well, is this project still being maintained?

mmmarchetti commented 1 year ago

hello godblessbb!

The team at AI4Finance is working to update these demos and examples to address the issues caused by the updates in the libraries used in the FinRL project. In your case, the error is caused by an update to the Gym environment. The issue will be resolved soon.

godblessbb commented 1 year ago

Hello Sir, thank you for your enthusiastic answers. I hope these demos can be fixed as soon as possible, because every step is very difficult, I encountered many puzzling errors, even doubt whether this project is still valid. For example, I am now stuck in the training step, I received an error:

AssertionError: The algorithm only supports <class 'gym.spaces.box.Box'> as action spaces but Box(-1.0, 1.0, (12,), float32) was provided.

It seems that my model(DDPG indeed) requires a Box type environment, but my training environment is Box type and cannot match the model. I really don't know where the problem is.

hello godblessbb!

The team at AI4Finance is working to update these demos and examples to address the issues caused by the updates in the libraries used in the FinRL project. In your case, the error is caused by an update to the Gym environment. The issue will be resolved soon.

godblessbb commented 1 year ago

hello godblessbb!

The team at AI4Finance is working to update these demos and examples to address the issues caused by the updates in the libraries used in the FinRL project. In your case, the error is caused by an update to the Gym environment. The issue will be resolved soon.

Solved again. Still due to the gym version upgrade, change the gymnasium module in env_stocktrading to the gym module and it works again. It was really tough.