containerd networking files are never cleaned up after the container exits - Githubissues

concourse / concourse

Concourse is a container-based continuous thing-doer written in Go.

https://concourse-ci.org

Apache License 2.0

7.37k stars 846 forks source link

containerd networking files are never cleaned up after the container exits #7267

Closed chenbh closed 3 years ago

chenbh commented 3 years ago

Reported on discord (https://discord.com/channels/219899946617274369/413770960089382922/862856028247556146)

Summary

The network files created by the containerd runtime is never cleaned up https://github.com/concourse/concourse/blob/c88bb501a05567ee40126d62f9b0e4776be31c06/worker/runtime/cni_network.go#L254

Steps to reproduce

Do something that will create a container
Wait for the container to exit
Look at the /tmp file

Expected results

No leftover files/dir

Actual results

The hosts, hostname, and resolv.conf file are left behind

Additional context

Hopping on one of our CI workers gave 27k folders in /tmp, and 272k on one of the older hush-house workers!

Also, should these files really be in /tmp? There's a reason we have CONCOURSE_WORK_DIR.

Triaging info

Concourse version:
Browser (if applicable):
Did this used to work?

taylorsilva commented 3 years ago

Potential workaround

The directories under /tmp are named after a container's handle. If you don't see a matching uuid when running fly containers or in the containers table in the db, then you can safely delete those directories.