search

kwai / DouZero

[ICML 2021] DouZero: Mastering DouDizhu with Self-Play Deep Reinforcement Learning | 斗地主AI
https://douzero.org/
Apache License 2.0
4.09k stars 588 forks source link

why use 'Lock' in "get_batch()" while using queue to communication between threads #22

Closed huzhoudaxia closed 3 years ago

huzhoudaxia commented 3 years ago

The codes is in douzero.dmc.utils, line 5 to 52. I am confused about the usage of lock. To my knowledge, queue module can achieve thread synchronization, then why use Lock mechanism again. I don't know whether my question is clearly. Hope it's worthy. Yours sincerely.

daochenzha commented 3 years ago

@huzhoudaxia Very good question. Basically, the get_batch is getting a batch of samples instead of one sample. After getting all the samples, it will send the signals to the free_queue. Since there are multiple threads, there could be some weird cases. Let me give you an example:

Suppose we use 4 threads, the batch size is 32, and the number of buffers is 50. The four threads all request 32 samples (and they will only send free signals after getting 32 samples). Suppose at some time step, they get, 12, 12, 12, and 14 samples, respectively. At this time step, the program will get stuck since the buffers have not been freed.

Thus, we add a lock here to make getting 32 samples atomic.

huzhoudaxia commented 3 years ago

@daochenzha Thank you for your time. Your answer is very helpful.

yffbit commented 3 years ago

similar to the Dining Philosophers Problem