search

nestybox / sysbox

An open-source, next-generation "runc" that empowers rootless containers to run workloads such as Systemd, Docker, Kubernetes, just like VMs.
Apache License 2.0
2.78k stars 152 forks source link

Sysbox-mgr fails leaks file-descriptor during container creation #114

Closed ctalledo closed 3 years ago

ctalledo commented 3 years ago

The sysbox-mgr has a file descriptor leak during container creation and/or removal.

This leak causes sysbox to fail to create containers after more than ~512 containers are created and removed, because this causes the default Linux limit for max opened file descriptors (1024) to be reached.

When the error occurs, we see the following in the sysbox-mgr log (/var/log/sysbox-mgr.log):

WARN[2020-11-09 19:59:25] sync-out for container 8b1f5eced84ca3ec1ded7b5bebfdaf32a0a490a57a7e84c2887936498994b6ea failed: sync-out for volume backing [var-lib-docker var-lib-kubelet var-lib-containerd-ovfs] failed: volume sync-out failed: failed to sync /var/lib/sysbox/containerd/8b1f5eced84ca3ec1ded7b5bebfdaf32a0a490a57a7e84c2887936498994b6ea/ to /var/lib/docker/165536.165536/overlay2/214855bdc8d28deed38e653fe9a780e7c1a519274e4b34c04613c829ba013170/merged/var/lib/containerd/io.containerd.snapshotter.v1.overlayfs:  fork/exec /usr/bin/rsync: too many open files
ctalledo commented 3 years ago

Fixed via these changes:

https://github.com/nestybox/sysbox-mgr/pull/10 https://github.com/nestybox/sysbox-libs/pull/10 https://github.com/nestybox/sysbox-runc/pull/16

Closing!