Support for VecVideoRecorder

[bragajj]

Support for VecVideoRecorder https://stable-baselines3.readthedocs.io/en/master/guide/vec_envs.html?highlight=video#vecvideorecorder

Hello, and thank you for the amazing procedurally generated environments.

I've experienced some issues recording videos using existing libraries such as https://github.com/DLR-RM/stable-baselines3. Specifically, SB3's VecVideoRecorder can only record the video if the vectorized environment has the metadata field, which is the first piece I've added for this PR. To fully record the video, it appears one must toggle the render_mode=rgb_array, but that unfortunately significantly slows down the throughput (going from 5k SPS to 400 SPS due to the overhead of rgb arrays in the infos), as is noted in several places prescribing this method of recording. So perhaps an current solution would be to also support returning the obs's rgb array through render('rgb_array') if we want fast throughput, which is the second part of this PR. We have tested out this approach and are able to record video during training and still keep fast throughput (see https://wandb.ai/cleanrl/cleanrl.benchmark/runs/kraizl7w as an example). It can be used in the following way

venv = ProcgenEnv(num_envs=64, env_name='starship', num_levels=0, start_level=0, distribution_mode='easy') venv = VecExtractDictObs(venv, "rgb") venv = VecMonitor(venv=venv) envs = VecNormalize(venv=venv, norm_obs=False) envs = VecPyTorch(envs, device) envs = VecVideoRecorder(envs, f'videos', record_video_trigger=lambda x: x % 1000000== 0, video_length=100)

[bragajj]

Hello, and thank you for the amazing procedurally generated environments.

I've experienced some issues recording videos using existing libraries such as https://github.com/DLR-RM/stable-baselines3. Specifically, SB3's VecVideoRecorder can only record the video if the vectorized environment has the metadata field, which is the first piece I've added for this PR. To fully record the video, it appears one must toggle the render_mode=rgb_array, but that unfortunately significantly slows down the throughput (going from 5k SPS to 400 SPS due to the overhead of rgb arrays in the infos), as is noted in several places prescribing this method of recording. So perhaps an current solution would be to also support returning the obs's rgb array through render('rgb_array') if we want fast throughput, which is the second part of this PR. We have tested out this approach and are able to record video during training and still keep fast throughput (see https://wandb.ai/cleanrl/cleanrl.benchmark/runs/kraizl7w as an example). It can be used in the following way

venv = ProcgenEnv(num_envs=64, env_name='starship', num_levels=0, start_level=0, distribution_mode='easy')
venv = VecExtractDictObs(venv, "rgb")
venv = VecMonitor(venv=venv)
envs = VecNormalize(venv=venv, norm_obs=False)
envs = VecPyTorch(envs, device)
envs = VecVideoRecorder(envs, f'videos', record_video_trigger=lambda x: x % 1000000== 0, video_length=100)

[vwxyzjn]

Maybe @kcobbe?

[christopherhesse]

This seems unnecessary for recording video with gym3. Are you still interested in merging this? I'm confused by the examples because you're using ProcgenEnv instead of ToBaselinesVecEnv, right?

[vwxyzjn]

This seems unnecessary for recording video with gym3.

Hey @christopherhesse, we are still interested in merging this. I think gym3 already has a video recording utility, but this PR enables video recording in SB3's vectorized environments that rely on the metadata.

I'm confused by the examples because you're using ProcgenEnv instead of ToBaselinesVecEnv, right?

We are using this:

def ProcgenEnv(num_envs, env_name, **kwargs):
    return ToBaselinesVecEnv(ProcgenGym3Env(num=num_envs, env_name=env_name, **kwargs))

[christopherhesse]

I forgot that the normal one was called ProcgenGym3Env. Okay, fix the fps and I will merge it.

[bragajj]

Env.py now updated to 15fps rather than 24fps in my branch in regards to comments

[christopherhesse]

Thanks!

[vwxyzjn]

Thank you @bragajj!