search

celery / kombu

Messaging library for Python.
http://kombu.readthedocs.org/
BSD 3-Clause "New" or "Revised" License
2.86k stars 926 forks source link

Possible Kombu bug matching corresponding Celery bug in Consul support. #1222

Open ShaheedHaque opened 4 years ago

ShaheedHaque commented 4 years ago

I noticed that Kombu has a similar pattern while initializing its Consul backend as Celery, as per the description and draft fix in celery/celery#5605. That is, it initializes the Consul client connection once like this:

https://github.com/celery/kombu/blob/016f4accb634ebc3eb8fc66741f23552e5966ea7/kombu/transport/consul.py#L59

and then uses it multiple times, without waiting for any given operation to return and thus execute the callback. This then means that a subsequent call can interpret the result of the earlier operation as the result it expects, with resulting chaos. The hack draft fix referred to celery/celery#5605 addresses the problem.

auvipy commented 3 years ago

where we stand on this now?

ShaheedHaque commented 3 years ago

I have not attempted a fix because I have not tried to use Consul as the broker. And I have not tried that because there is nothing in the docs to indicate Consul is a possible broker backend. I'd certainly be interested in using it over RabbitMQ if possible...?