search

sni / mod_gearman

Distribute Naemon Host/Service Checks & Eventhandler with Gearman Queues. Host/Servicegroups affinity included.
http://www.mod-gearman.org
GNU General Public License v3.0
122 stars 42 forks source link

On gearmand side, connections to workers stay in CLOSE_WAIT state eventually consuming all open filehandles #46

Closed smetj closed 10 years ago

smetj commented 11 years ago

Gearmand 0.25 Mod_gearman: 1.4.6 OS: rhel 6.2 Kernel 2.6.32-220.17.1.el6.x86_64

lsof command output

gearmand 16611 gearmand 9947u IPv4 3479410565 0t0 TCP master-101:gearman->worker-205:51992 (CLOSE_WAIT) gearmand 16611 gearmand 9950u IPv4 3479426713 0t0 TCP master-101:gearman->worker-102:42700 (CLOSE_WAIT) gearmand 16611 gearmand 9951u IPv4 3479365897 0t0 TCP master-101:gearman->worker-203:59654 (CLOSE_WAIT)

When all filehandles are consumed, the possibility to create a new connection to gearmand is sparse (until close_wait) times out.

It seems gearmand is not closing the socket when a client disconnects? This brings me to another topic, I wonder why the clients disconnect so often, ... we don't seem to have network issues yet, mod_gearman clients produce many of these: "[2013-09-01 22:36:38][31858][ERROR] sending job to gearmand failed: libgearman/connection.cc:482(GEARMAN_GETADDRINFO) Name or service not known -> getaddrinfo:%s"

sni commented 11 years ago

I cannot reproduce that with the current stable release. However there was a problem with CLOSE_WAIT connection in previous releases, so i assume this is already fixed. I don't think the dns errors are related to Mod-Gearman or Gearman itself.

smetj commented 11 years ago

I based myself upon this statement in the documentation: "Latest stable version 1.4.2, released January 10 2013" https://labs.consol.de/nagios/mod-gearman/#_download

This piece of the documentation should be updated then as the latest stable is a newer version than indicated.

I'll upgrade to the latest version and evaluate

sni commented 11 years ago

Thanks, i just updated the page to reflect latest updates.