search

mosquito / aio-pika

AMQP 0.9 client designed for asyncio and humans.
https://aio-pika.readthedocs.org/
Apache License 2.0
1.18k stars 186 forks source link

aio-pika stuck in infinite loop #565

Closed isra17 closed 11 months ago

isra17 commented 11 months ago

I've been experiencing 100% CPU usage and whole process hanging lately. After some more investigation I found it comes from aio-pika under certain scenario.

I'm able to reliably reproduce by declaring a RobustQueue with await robust_channel.declare_queue(...) that fails when the queue options change, getting the error ChannelPreconditionFailed: PRECONDITION_FAILED - inequivalent arg 'x-max-length-bytes' for queue 'repro-stream' in vhost '/': received '100' but current is '10'

Being able to attach to the process I see the thread looping in aio_pika/robust_channel.py(77)ready() where self.__restored is False and self.__ready is set so the wait() returns right away without yielding.

Looking at the code, it seems like there's a window where __ready is set when __restored is still False.

Will try to open a PR with a fix.

mosquito commented 11 months ago

Is possible to restoring with passive=True argument but really, what when you program has been restarted?

I still don't understand how this happened, how during the reconnection time someone redefined a queue with the same name but different arguments.