search

MorvanZhou / pytorch-A3C

Simple A3C implementation with pytorch + multiprocessing
https://mofanpy.com
MIT License
614 stars 142 forks source link

question about push_and_pull funtion #6

Closed luochao1024 closed 6 years ago

luochao1024 commented 6 years ago

Hi, Morvan,

In your push_and_pull function, you update the global grad without any condition, see https://github.com/MorvanZhou/pytorch-A3C/blob/master/discrete_A3C.py#L95. However, for other implementations of a3c which are quite similar with yours update global grad only when global grad is None. See https://github.com/greydanus/baby-a3c/blob/master/baby-a3c.py#L159 and https://github.com/ikostrikov/pytorch-a3c/blob/master/train.py#L13. I am quite confused about it. Would you mind giving me your insight about it?

MorvanZhou commented 6 years ago

Hi Luochao,

The reason they write it that way might be to keep the update thread safe. In my application, the global parameter can be updated while sharing with local workers or updated by many workers at the same time. You can try to add a lock to the updating step to maintain a sequential updating.

I have tried adding a lock, but from what I see, there is not too much improvement. I think without a lock, the updating mechanism would be like the mechanism in this paper "Hogwild: A lock-free approach to parallelizing stochastic gradient descent"