search

MetPX / sarracenia

https://MetPX.github.io/sarracenia
GNU General Public License v2.0
45 stars 22 forks source link

socket leak when connecting to non-existent exchange. #53

Open petersilva opened 6 years ago

petersilva commented 6 years ago

import of: https://sourceforge.net/p/metpx/bugs/48/

2016-08-17... Murray reports:

Reported by Jun on Monday... He noticed that sr_shovel, when connecting to a broker and attempting to bind to a non-existent exchange, will continously retry, likely not closing the previous attempts. this causes socket descriptors on the broker to be consumed eventually leading to system-wide exhaustion on the broker.

(incident 2016/08/15 02h00 Z... )

This is likely a problem in sr_consumer or sr_amqp that would affect all components.

petersilva commented 6 years ago

MR 2017/08/17: When sr_shovel attempts to connect to a non-existant exchange and fails, it then defaults to the default exchange. This is the "(AMQP Default)" exchange, and this is what the queue is attached to.

What I found when I tried to conenct to a non-existant exchange was that it just sat there after failing to connect probably because it had then connected to the default exchange.

petersilva commented 6 years ago

I tested shovelling to a nonexistant exchange. No increase in sockets.

petersilva commented 6 years ago

While I had a shovel shovelling from one broker to another I deleted the receining broker. No difference

petersilva commented 6 years ago

Dried killing an exchange which had one shovel feeding it and another feeding off of it. No new sockets.

petersilva commented 6 years ago

I set up a shovel that reads from a non-existant exchange and writes to a non-existent exchange. The log show it retrying to connect to the non-existant exchange once every 5 seconds. But the socket descriptors are not increasing.

petersilva commented 6 years ago

chat with Jun to see if on_poll scripts are causing the issue (try using his...)

petersilva commented 6 years ago

sr_shovel doesn't use on_poll scripts