search

Khrylx / PyTorch-RL

PyTorch implementation of Deep Reinforcement Learning: Policy Gradient methods (TRPO, PPO, A2C) and Generative Adversarial Imitation Learning (GAIL). Fast Fisher vector product TRPO.
MIT License
1.09k stars 186 forks source link

Question on multiprocessing #22

Closed pengzhi1998 closed 4 years ago

pengzhi1998 commented 4 years ago

Hi @Khrylx, thank you so much for this great repository! But I found in the Readme:

PyTorch will create additional threads when performing computations which can damage the performance of multiprocessing. This problem is most serious with Linux, where multiprocessing can be even slower than a single thread

and found here you only set one process. I'm a little bit confused, do you recommend using multiple processes to train the network? But if it only uses one process to train the network, what's its advantage over memory buffer? Thank you!

Khrylx commented 4 years ago

The reason for setting OMP_NUM_THREADS=1 is because when we create multiple worker threads to sample trajectories, we want each worker thread to only use its own thread instead of taking resources from other threads, which can slow down the overall speed.

Also, for the code you referred to, I am actually using "self.num_threads" number of threads, which is a parameter to the agent.