search

vwxyzjn / cleanrl

High-quality single file implementation of Deep Reinforcement Learning algorithms with research-friendly features (PPO, DQN, C51, DDPG, TD3, SAC, PPG)
http://docs.cleanrl.dev
Other
5.54k stars 631 forks source link

AsyncVectorEnv #244

Closed Jogima-cyber closed 2 years ago

Jogima-cyber commented 2 years ago

Hello, I was wondering why not use AsyncVectorEnv instead of SyncVectorEnv ? I don't see any reasons trajectories should not be parallelized.

vwxyzjn commented 2 years ago

Good question! SyncVectorEnv is just easier to debug, but we can definitely use AsyncVecEnv.

Jogima-cyber commented 2 years ago

Thanks, just wanted to have your opinion on the subject. I will run some tests with it then.

vwxyzjn commented 2 years ago

FYI - if you want to add it, feel free to submit a PR incorporating a flag like this:

https://github.com/vwxyzjn/vectorized-value-methods/blob/65b4745198c77d9b252ac90f373d9ad019003d9f/ppo_continuous_action.py#L48-L49