[Bug]: video recording on Pybullet

[guillaumepourcel]

🐛 Bug

After training a PyBullet environment, I try to record a video of it:

python -m rl_zoo3.record_video --algo sac --env Walker2DBulletEnv-v0 --exp-id 10 -f logs/ -n 100

And I get the following output:

pybullet build time: May 20 2022 19:45:31
Loading logs/sac/Walker2DBulletEnv-v0_10/Walker2DBulletEnv-v0.zip
Traceback (most recent call last):
  File "/home/guillaumep/miniconda3/envs/zoo_test/lib/python3.9/runpy.py", line 197, in _run_module_as_main
    return _run_code(code, main_globals, None,
  File "/home/guillaumep/miniconda3/envs/zoo_test/lib/python3.9/runpy.py", line 87, in _run_code
    exec(code, run_globals)
  File "/home/guillaumep/rl-baselines3-zoo/rl_zoo3/record_video.py", line 141, in <module>
    env = VecVideoRecorder(
  File "/home/guillaumep/miniconda3/envs/zoo_test/lib/python3.9/site-packages/stable_baselines3/common/vec_env/vec_video_recorder.py", line 50, in __init__
    assert self.env.render_mode == "rgb_array", f"The render_mode must be 'rgb_array', not {self.env.render_mode}"
AssertionError: The render_mode must be 'rgb_array', not None
Exception ignored in: <function VecVideoRecorder.__del__ at 0x7fb1ffb8ca60>
Traceback (most recent call last):
  File "/home/guillaumep/miniconda3/envs/zoo_test/lib/python3.9/site-packages/stable_baselines3/common/vec_env/vec_video_recorder.py", line 113, in __del__
    self.close_video_recorder()
  File "/home/guillaumep/miniconda3/envs/zoo_test/lib/python3.9/site-packages/stable_baselines3/common/vec_env/vec_video_recorder.py", line 103, in close_video_recorder
    if self.recording:
  File "/home/guillaumep/miniconda3/envs/zoo_test/lib/python3.9/site-packages/stable_baselines3/common/vec_env/base_vec_env.py", line 392, in __getattr__
    return self.getattr_recursive(name)
  File "/home/guillaumep/miniconda3/envs/zoo_test/lib/python3.9/site-packages/stable_baselines3/common/vec_env/base_vec_env.py", line 417, in getattr_recursive
    attr = getattr(self.venv, name)
AttributeError: 'SubprocVecEnv' object has no attribute 'recording'```

### To Reproduce

_No response_

### Relevant log output / Error message

_No response_

### System Info

- OS: Linux-5.15.90.1-microsoft-standard-WSL2-x86_64-with-glibc2.31 # 1 SMP Fri Jan 27 02:56:13 UTC 2023
- Python: 3.9.18
- Stable-Baselines3: 2.1.0
- PyTorch: 2.1.0
- GPU Enabled: True
- Numpy: 1.26.0
- Cloudpickle: 3.0.0
- Gymnasium: 0.29.1
- OpenAI Gym: 0.26.2
- Describe how Stable Baselines3 was installed: pip install -e

I also got the same error with the colab: [RL Baselines zoo](https://colab.research.google.com/github/Stable-Baselines-Team/rl-colab-notebooks/blob/sb3/rl-baselines-zoo.ipynb)

### Checklist

- [X] I have checked that there is no similar [issue](https://github.com/DLR-RM/rl-baselines3-zoo/issues) in the repo
- [X] I have read the [SB3 documentation](https://stable-baselines3.readthedocs.io/en/master/)
- [X] I have read the [RL Zoo documentation](https://rl-baselines3-zoo.readthedocs.io)
- [X] I have provided a [minimal and working](https://github.com/DLR-RM/stable-baselines3/issues/982#issuecomment-1197044014) example to reproduce the bug
- [X] I've used the [markdown code blocks](https://help.github.com/en/articles/creating-and-highlighting-code-blocks) for both code and stack traces.

[araffin]

Hello, probably a duplicate of https://github.com/DLR-RM/rl-baselines3-zoo/issues/399, please use SB3 master branch (see doc), I'll release a v2.2.0 soon.

[guillaumepourcel]

Hello, Indeed, it works now for the example of #399 with the swimmer-v3, but not for Pybullet environments (I tested with the environment Walker2DBulletEnv-v0). Maybe because Pybullet environments are using SubprocVecEnv instead of DummyVecEnv?

[araffin]

The issue comes from https://github.com/araffin/pybullet_envs_gymnasium, not SB3, the render mode is not properly set, I will try to push a fix soon.

[araffin]

pip install pybullet_envs_gymnasium --upgrade should fix the issue ;) (I'm also removing old hack from the RL Zoo in https://github.com/DLR-RM/rl-baselines3-zoo/pull/421)