search

celery / kombu

Messaging library for Python.
http://kombu.readthedocs.org/
BSD 3-Clause "New" or "Revised" License
2.86k stars 927 forks source link

socket leak in redis-sentinel #1108

Open himanshu-nskp opened 4 years ago

himanshu-nskp commented 4 years ago

I have been experiencing an issue where kombu-sentinel support is leaking the socket opened to redis-sentinel nodes. Kombu-sentinel keeps reconnecting the redis sentinel nodes and leaving the connected sockets behind, If there is network glitch or some other known-unknown issue which is preventing kombu-sentinel to connect to redis master node

Reproduction: when celery is configured with different "mastername" in comparison to what redis-sentinel setups are configured with.
For ex: Redis sentinel setup has master as "masternode" but the celery broker is configured as "masternode1". This way kombu-sentinel support will not be able to find the redis master node and keep reconnecting and leaving the connected sentinel sockets behind.

Redis sentinel will reach to "max connected client" eventually and the complete system will be dead.

himanshu-nskp commented 4 years ago

open_source_patch.txt I tried this patch and it smoothly working for me. Now it is disconnecting the sentinel sockets properly.

thedrow commented 4 years ago

Please submit a PR. It's not possible to review a patch this way.