search

celery / billiard

Multiprocessing Pool Extensions
Other
419 stars 252 forks source link

Queue put-get delay of many seconds? #233

Closed mike-lawrence closed 7 years ago

mike-lawrence commented 7 years ago

I'm using billiard to run two processes (a & b) in parallel with a pair of Queue()s to communicate between them (a->b and b->a) and despite using a loop in each process that should be checking the queues every few milliseconds, I'm experiencing a delay of up to many seconds between one process putting a message in a queue and the other process receiving it. Frustratingly, it seems to be variable as to when these delays manifest, with delays of only 10-100ms otherwise. Any suggestions for troubleshooting something like this?

mike-lawrence commented 7 years ago

Closing because, while I've made it reliably reproducible, I think I've narrowed this down to another library I use.