Closed vcharraut closed 1 year ago
The latest updates on your projects. Learn more about Vercel for Git ↗︎
Name | Status | Preview | Comments | Updated (UTC) |
---|---|---|---|---|
cleanrl | ✅ Ready (Inspect) | Visit Preview | 💬 Add feedback | May 2, 2023 8:03pm |
The error is due to needing stable baselines 3 ==2
No sign of regression, as shown in the PR description. Merging now.
Hi @vwxyzjn @charraut, I'm wondering what part of this change forced us to add the following line:
assert args.num_envs == 1, "vectorized envs are not supported at the moment"
Vectorization was a useful feature earlier. Thank you!
@ronuchit this is due to SB3's replay buffer don't support num_envs>1
I think.
I believe it does, actually: https://github.com/DLR-RM/stable-baselines3/blame/master/stable_baselines3/common/buffers.py#L162
We would just need to pass in n_envs=args.num_envs
when we instantiate the ReplayBuffer. Perhaps there are other issues at play here?
I believe it does, actually: https://github.com/DLR-RM/stable-baselines3/blame/master/stable_baselines3/common/buffers.py#L162
We would just need to pass in
n_envs=args.num_envs
when we instantiate the ReplayBuffer. Perhaps there are other issues at play here?
I see. That’s interesting. Would you be interested in making a PR that optionally supports num_envs>1?
sure, done: https://github.com/vwxyzjn/cleanrl/pull/395
Description
This PR updates the DQN files to the lastest version of gymnasium, replacing gym.
dqn.py
dqn_jax.py
dqn_atari.py
dqn_atari_jax.py
Types of changes
Checklist:
pre-commit run --all-files
passes (required).mkdocs serve
.If you need to run benchmark experiments for a performance-impacting changes:
--capture-video
.python -m openrlbenchmark.rlops
.python -m openrlbenchmark.rlops
utility to the documentation.python -m openrlbenchmark.rlops ....your_args... --report
, to the documentation.Regression report
https://wandb.ai/costa-huang/cleanrl/reports/Regression-Report-dqn_atari_jax--Vmlldzo0MjQ5OTA2
https://wandb.ai/costa-huang/cleanrl/reports/Regression-Report-dqn_jax--Vmlldzo0MjUwMDM1