search

CastawayLabs / cachet-monitor

Distributed monitoring plugin for CachetHQ
https://castawaylabs.github.io/cachet-monitor/
MIT License
439 stars 127 forks source link

socket: too many open files #75

Closed Porcupine96 closed 5 years ago

Porcupine96 commented 7 years ago

I'm having an issue - the cachet-monitor seems to leave the sockets opened.

At the beginning everything works fine but the number of used file descriptors constantly increases till it reaches the soft limit (1024) then I'm getting an error:

... level=warning msg="Could not log metric! ... socket: too many open files

Thank you for your time,

davidbogdanprivate commented 6 years ago

I have the same issue, somebody, please comment about it.

jdegger commented 6 years ago

Same here, this seems to be a fairly important problem. Any solutions?

samwiseg0 commented 6 years ago

same problem. It seems it never closes the socket when it checks.

srabouin commented 6 years ago

I found the issue, see below :)

RubenFixit commented 5 years ago

I'm seeing this issue again using the per-compiled binarys (cachet_monitor_linux_amd64 and cachet_monitor_linux_386) with v3.0.

Ran this on two separate systems: Linux 4.14.67-grsec-grsec+ #4 SMP Thu Sep 6 01:23:09 PDT 2018 x86_64 x86_64 x86_64 GNU/Linux and Linux 2.6.32-954.3.5.lve1.4.59.el6.x86_64 #1 SMP Thu Dec 6 05:11:00 EST 2018 x86_64 x86_64 x86_64 GNU/Linux

romeritoCL commented 5 years ago

Same issue here

time="2019-09-05T18:36:23Z" level=warning msg="Could not log metric! ID: 2, err: Post https://example.com/api/v1/metrics/2/points: dial tcp: lookup example.com on: dial udp: socket: too many open files"

voarsh2 commented 4 years ago

Me too

voarsh2 commented 4 years ago

Oh I see, the binary that has this issue fixed wasn't released......

voarsh2 commented 4 years ago

Can anyone help me build the cachet-monitor binary to include the latest commits in master to fix this issue?

romeritoCL commented 4 years ago

I added to the crontab a restart of the service each night, that fixed the issue. It's not the best solution ever, but given the time I had, it was a working and effective solution.

RubenFixit commented 4 years ago

I've been restarting every hour with a crontab because of the php memory limitation of the shared host I'm using.