search

draios / sysdig

Linux system exploration and troubleshooting tool with first class support for containers
http://www.sysdig.com/
Other
7.79k stars 728 forks source link

UDP connections not closing anymore #2076

Closed kayscheerer closed 8 months ago

kayscheerer commented 8 months ago

We've traced back a problem with UDP connections to the sysdig agent: We are running a sysdig-agent on a virtual machine running Red Hat Enterprise Linux release 8.9 (Ootpa)

dnf list installed | grep draios
draios-agent.x86_64                           13.0.0-1                                  @draios

The file is installed via the bash script from https://ibm.biz/install-sysdig-agent followed by dragent_installer.sh -a <<token>> -c ingest.eu-de.monitoring.cloud.ibm.com --collector_port 6443 --secure true -ac 'sysdig_capture_enabled: false' -ac 'prometheus: enabled: true' We have three different VMs, two are running version 13.0.0 and one is running version 12.8.0 Version 12.8 does not have this problem. Disabling the draios agent resolves the problem.

We can see that all (UDP) connections stay open, mostly do a DNS server.

with netstat -alnp | grep udp we can see that no process ID, example line:

... udp 0 0 <>:44259 <>:53 ESTABLISHED - udp 0 0 <>:44260 <>:53 ESTABLISHED - udp 0 0 <>:44261 <>:53 ESTABLISHED - ...

the number of open UDP calls (wc -l /proc/net/udp) increases every time we make any network related call. At some point, all sockets are exhausted and at this point only restarting the VM works.

I tried downgrading the module back to dragent 12.8.0 but did not find a way over dnf or the dragent installer file.

therealbobo commented 8 months ago

Hey @kayscheerer! Thanks for the issue but this repository is meant for the sysdig cli tool project and not for the Sysdig Agent project. To solve your issue, please try to upgrade the package (see this page for the changelog https://docs.sysdig.com/en/docs/release-notes/sysdig-agent-release-notes/#1301-march-11-2024) and/or contact the Sysdig support. 😄