NVIDIA / enroot

A simple yet powerful tool to turn traditional container/OS images into unprivileged sandboxes.
Apache License 2.0
649 stars 94 forks source link

Empty directory missing in imported Docker image #170

Closed jfolz closed 10 months ago

jfolz commented 11 months ago

While attempting to import & run a Selenium image from Docker, I was greeted with a message about a missing directory:

$ enroot import -o selenium_standalone-chrome_120.0.sqsh docker://selenium/standalone-chrome:120.0
$ enroot create -n selenium -- selenium_standalone-chrome_120.0.sqsh
[INFO] Extracting squashfs filesystem...
Parallel unsquashfs: Using 40 processors
31909 inodes (32611 blocks) to write

created 24013 files
created 3477 directories
created 7875 symlinks
created 0 devices
created 0 fifos
created 0 sockets
$ enroot start selenium
Error: The directory named as part of the path /var/run/supervisor/supervisord.pid does not exist
For help, use /usr/bin/supervisord -h

Verifying inside the container, the directory is indeed missing:

$ enroot start test bash
# ls -l /var/run/supervisor
ls: cannot access '/var/run/supervisor': No such file or directory

Using podman to start a container, the directory exists:

$ podman run --rm -it docker://selenium/standalone-chrome:120.0 bash
$ ls -l /var/run/supervisor
total 0

This is our enroot.conf:

ENROOT_CONFIG_PATH ${HOME}/.config/enroot
ENROOT_SQUASH_OPTIONS -b 1048576 -comp zstd -Xcompression-level 2
ENROOT_MOUNT_HOME n
ENROOT_RESTRICT_DEV y
ENROOT_REMAP_ROOT y
ENROOT_ROOTFS_WRITABLE y
ENROOT_RUNTIME_PATH /run/enroot-runtime/user-$(id -u)
ENROOT_CACHE_PATH /var/cache/enroot-cache/group-$(id -g)
ENROOT_DATA_PATH /run/enroot-data/user-$(id -u)
flx42 commented 11 months ago

enroot mounts a tmpfs on /var/run by default, to be similar to a bare-metal system. Thus hiding what the container image stores in these folders. Container images writers should probably avoid creating files/directories in folders like /var/run or /tmp.

3XX0 commented 11 months ago

Note that if you don't want this behavior you can always edit the enroot mount configuration. Alternatively you can mount a tmpfs when you start the container, or create the directory in your entrypoint:

enroot start -m tmpfs:/run/supervisor:x-create=dir,mode=0600 selenium

jfolz commented 11 months ago

@flx42 Interesting. I wasn't aware of this, but I understand the reasoning. I suppose bare metal these folders are created by tmpfiles.d or some similar mechanism, which isn't effective in a container. In this case, mkdir /var/run/supervisor fixes the issue, which is simple enough. TBH, this is the first time we've encountered such an issue in 3+ years, so I don't know whether it's even worth looking into it much. My only concern is confused users coming to me with this type of issue ;)

jfolz commented 10 months ago

Happy new year! We decided to install Podman alongside Enroot on compute nodes. This not only allows users to build images on-site (and then enroot import them), but we can also point people in that direction if something like this crops up. Given the rarity and existing workaround, I'm closing this issue.