search

openai / baselines

OpenAI Baselines: high-quality implementations of reinforcement learning algorithms
MIT License
15.7k stars 4.87k forks source link

A2C multicore usage is not parallel #195

Closed shanlior closed 6 years ago

shanlior commented 6 years ago

When running A2C , the envelopes does not run at a parallel manner as can be seen from the ubuntu system monitor. This hurts the performance, and makes A3C an even choice, even while using GTX 1080ti. I'm using Intel i7-7700 and a dual GTX 1080ti setup. (I also tried to disable one of the GPU'S with CUDA_VISIBLE_DEVICES)

image

olegklimov commented 6 years ago

Please check again if it is the case. I've found from looking at the code, that run_atari.py for a2c uses num_env=16, so you should see 16 python processes using CPU simultaneously. Then a lot of GPU usage, then CPUs again.