search

mesosphere / marathon-lb

Marathon-lb is a service discovery & load balancing tool for DC/OS
Apache License 2.0
449 stars 300 forks source link

HAProxy: [ALERT] sendmsg logger #2 failed: Resource temporarily unavailable (errno=11) #488

Open ayashjorden opened 7 years ago

ayashjorden commented 7 years ago

Hi, Opening here as the error message is coming from the bundled HAProxy component in MLB container.

Context:

When running MLB without enabling the /devlog device for HAProxy, I'm getting blasted with: [ALERT] 228/081350 (412) : sendmsg logger #2 failed: No such file or directory (errno=2)

After enabling the /dev/log device:

  1. I've started seeing HAProxy log messages
  2. I'm getting blasted with: [ALERT] 228/081044 (10579) : sendmsg logger #2 failed: Resource temporarily unavailable (errno=11)

I've searched online for similar solutions but all I found is not MLB related. Tried restarting the rsyslog daemon, didn't help.

UPDATE: Found this issue which is related.

Questions

Can someone shed some light here? Why mounting the /dev/log device into the MLB container didn't help? Can someone elaborate on how to configure rSyslog on the host machine to support HAProxy logging to /dev/log ?

Running on CentOS7.3.1611

Thank you, Yarden

petr-k commented 6 years ago

We're also facing the same issues.. would be cool if somebody could explain what the correct configuration should look like. Thanks! ;)

sureshcvh commented 6 years ago
  1. Increased the Max datagram queue length (/proc/sys/net/unix/max_dgram_qlen via sysctl --> sysctl -w net.unix.max_dgram_qlen=100) from 10 to 100.

and

2.restarted the syslog (sudo service rsyslog restart).

Alerts stopped showing.

treborbg commented 3 years ago

In my case syslog service was faulty, I restart it and I was able to move on. service syslog restart