Open vwxyzjn opened 1 year ago
The latest updates on your projects. Learn more about Vercel for Git ↗︎
Name | Status | Preview | Updated |
---|---|---|---|
cleanrl | ✅ Ready (Inspect) | Visit Preview | Jan 13, 2023 at 2:25PM (UTC) |
Thank you @vwxyzjn! I'll give this a spin.
hello. Thanks alot for implementing PPO in JAX in such a clean fashion. But, while reproducing the results, i am facing the following issue.
Traceback (most recent call last):
File "/scratch/vaidya/mujoco_sims/gym_mujoco_drones/gym_mujoco_drones/cleanrl_jax_ppo.py", line 199, in <module>
agent_state = TrainState.create(
File "/scratch/vaidya/miniconda3/envs/sbx-gpu/lib/python3.10/site-packages/flax/training/train_state.py", line 127, in create
params['params'] if OVERWRITE_WITH_GRADIENT in params else params
TypeError: argument of type 'AgentParams' is not iterable
Exception ignored in: <function AsyncVectorEnv.__del__ at 0x7f6aa6d89630>
Traceback (most recent call last):
File "/scratch/vaidya/miniconda3/envs/sbx-gpu/lib/python3.10/site-packages/gymnasium/vector/async_vector_env.py", line 549, in __del__
File "/scratch/vaidya/miniconda3/envs/sbx-gpu/lib/python3.10/site-packages/gymnasium/vector/vector_env.py", line 272, in close
File "/scratch/vaidya/miniconda3/envs/sbx-gpu/lib/python3.10/site-packages/gymnasium/vector/async_vector_env.py", line 465, in close_extras
AttributeError: 'NoneType' object has no attribute 'TimeoutError'
Since i am currently new to JAX, i am unable to debug the issue of AgentParams
being not iterable
on my own. I understand that this is a work in progress, but i would appreciate any pointers to solve this.
Thanks
Description
This PR modifies
ppo_continuous_action.py
to make it more parallel-envs-friendly. CC @kevinzakka.The version of
ppo_continuous_action.py
in this PR is different from that in themaster
branch in the following ways:gym.vector.AsyncVectorEnv
in favor ofgym.vector.SyncVectorEnv
to speed up things more https://github.com/vwxyzjn/cleanrl/blob/703cd3ba1214a15d2fc6ce9157f8c094d627c07b/cleanrl/ppo_continuous_action.py#L163-L165I also added a JAX variant that reached the same level of performance
Types of changes
Checklist:
pre-commit run --all-files
passes (required).mkdocs serve
.If you are adding new algorithm variants or your change could result in performance difference, you may need to (re-)run tracked experiments. See https://github.com/vwxyzjn/cleanrl/pull/137 as an example PR.
--capture-video
flag toggled on (required).mkdocs serve
.