[Bug]: Scaling Environment

[Hamza-101]

🐛 Bug

I have trained and tested a custom Boid flocking environment, in OpenAI Gym, using 3 Boids it works. However when I test it, the policy, for greater than that e.g 10, it gives the following error.

I am essentially trying to create an extensible policy for boid flocking.

To Reproduce

Download Code: https://drive.google.com/drive/folders/1c0t-7D5RWumtLY9Bh9kht6P3RAQ4mq0V?usp=sharing Download Model: https://drive.google.com/file/d/1wrBZ6mSrcaxrWERgvUYA_vLDaA0JL08f/view?usp=drive_link

Steps:

1. Place the model with code
2. cd into where code is and run command:

python PPONew.py

it will output error as given below.

Relevant log output / Error message

> PS D:\Test> python PPonew.py
> 2024-04-23 12:21:14.425966: I tensorflow/core/util/port.cc:113] oneDNN custom operations are on. You may see slightly different numerical results due to floating-point round-off errors from different computation orders. To turn them off, set the environment variable `TF_ENABLE_ONEDNN_OPTS=0`.
> 2024-04-23 12:21:15.190142: I tensorflow/core/util/port.cc:113] oneDNN custom operations are on. You may see slightly different numerical results due to floating-point round-off errors from different computation orders. To turn them off, set the environment variable `TF_ENABLE_ONEDNN_OPTS=0`.
> C:\Users\Cr7th\AppData\Local\Programs\Python\Python311\Lib\site-packages\stable_baselines3\common\save_util.py:166: UserWarning: Could not deserialize object clip_range. Consider using `custom_objects` argument to replace this object.
> Exception: Can't get attribute '_make_function' on <module 'cloudpickle.cloudpickle' from 'C:\\Users\\Cr7th\\AppData\\Local\\Programs\\Python\\Python311\\Lib\\site-packages\\cloudpickle\\cloudpickle.py'>
>   warnings.warn(
> C:\Users\Cr7th\AppData\Local\Programs\Python\Python311\Lib\site-packages\stable_baselines3\common\save_util.py:166: UserWarning: Could not deserialize object lr_schedule. Consider using `custom_objects` argument to replace this object.
> Exception: Can't get attribute '_make_function' on <module 'cloudpickle.cloudpickle' from 'C:\\Users\\Cr7th\\AppData\\Local\\Programs\\Python\\Python311\\Lib\\site-packages\\cloudpickle\\cloudpickle.py'>
>   warnings.warn(
>   0%|                                                                                                                                                                                                           | 0/5 [00:00<?, ?it/s]Episode: 0
>   0%|                                                                                                                                                                                                           | 0/5 [00:00<?, ?it/s] 
> Traceback (most recent call last):
>   File "D:\Thesis_\Test\PPonew.py", line 499, in <module>
>     action, state = model.predict(obs)
>                     ^^^^^^^^^^^^^^^^^^
>   File "C:\Users\Cr7th\AppData\Local\Programs\Python\Python311\Lib\site-packages\stable_baselines3\common\base_class.py", line 553, in predict
>     return self.policy.predict(observation, state, episode_start, deterministic)
>            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
>   File "C:\Users\Cr7th\AppData\Local\Programs\Python\Python311\Lib\site-packages\stable_baselines3\common\policies.py", line 363, in predict
>     obs_tensor, vectorized_env = self.obs_to_tensor(observation)
>                                  ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
>   File "C:\Users\Cr7th\AppData\Local\Programs\Python\Python311\Lib\site-packages\stable_baselines3\common\policies.py", line 270, in obs_to_tensor
>     vectorized_env = is_vectorized_observation(observation, self.observation_space)
>                      ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
>   File "C:\Users\Cr7th\AppData\Local\Programs\Python\Python311\Lib\site-packages\stable_baselines3\common\utils.py", line 399, in is_vectorized_observation
>     return is_vec_obs_func(observation, observation_space)  # type: ignore[operator]
>            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
>   File "C:\Users\Cr7th\AppData\Local\Programs\Python\Python311\Lib\site-packages\stable_baselines3\common\utils.py", line 266, in is_vectorized_box_observation
>     raise ValueError(

> ValueError: Error: Unexpected observation shape (10, 4) for Box environment, please use (3, 4) or (n_env, 3, 4) for the observation shape.

System Info

sb3.get_system_info()

• OS: Windows-10-10.0.22631-SP0 10.0.22631
• Python: 3.11.4
• Stable-Baselines3: 2.2.1
• PyTorch: 2.0.0+cpu
• GPU Enabled: False
• Numpy: 1.23.5
• Cloudpickle: 1.2.2
• Gymnasium: 0.29.1
• OpenAI Gym: 0.15.7

({'OS': 'Windows-10-10.0.22631-SP0 10.0.22631', 'Python': '3.11.4', 'Stable-Baselines3': '2.2.1', 'PyTorch': '2.0.0+cpu', 'GPU Enabled': 'False', 'Numpy': '1.23.5', 'Cloudpickle': '1.2.2', 'Gymnasium': '0.29.1', 'OpenAI Gym': '0.15.7'}, '- OS: Windows-10-10.0.22631-SP0 10.0.22631\n- Python: 3.11.4\n- Stable-Baselines3: 2.2.1\n- PyTorch: 2.0.0+cpu\n- GPU Enabled: False\n- Numpy: 1.23.5\n- Cloudpickle: 1.2.2\n- Gymnasium: 0.29.1\n- OpenAI Gym: 0.15.7\n')

Checklist

• [ ] My issue does not relate to a custom gym environment. (Use the custom gym env template instead)
• [X] I have checked that there is no similar issue in the repo
• [X] I have read the documentation
• [X] I have provided a minimal and working example to reproduce the bug
• [X] I've used the markdown code blocks for both code and stack traces.

[qgallouedec]

Link #1905

I have trained and tested a custom Boid flocking environment

• [x] My issue does not relate to a custom gym environment. (Use the custom gym env template instead)

Please complete your message with the missing elements (minimal code among others) Have you tried the wrapper mentioned?

[Hamza-101]

Link #1905

I have trained and tested a custom Boid flocking environment

• [x] My issue does not relate to a custom gym environment. (Use the custom gym env template instead)

Please complete your message with the missing elements (minimal code among others) Have you tried the wrapper mentioned?

Sorry I don't exactly get what's required. Which wrapper, Can you please clarify? 👍

[qgallouedec]

Check the custom environment issue template. In future, read the instructions carefully (eg, don't check a box if you haven't read the associated item), they are there to help you and to help us help you.

[Hamza-101]

Sorry, just unchecked it.

[Hamza-101]

Found what to do. Sorry for the hassle.

[qgallouedec]

Good. However, this is not technical support or after-sales service. Thanks for considering posting your solution so that it can benefit everyone.

[Hamza-101]

Yes I am not considering as such. Is the last sentence sarcasm?

[qgallouedec]

No, the aim of the issues is for people to be able to ask their question and for the answer to help future people with the same problem. As a result, asking a question and closing it without sharing the solution is not good practice.

[Hamza-101]

So yeah. Yeah I get it.