NVIDIA / enroot

A simple yet powerful tool to turn traditional container/OS images into unprivileged sandboxes.
Apache License 2.0
648 stars 94 forks source link

`enroot-mount: failed to create file` #152

Closed ltalirz closed 1 year ago

ltalirz commented 1 year ago

This is with enroot 3.4.0 on CentOS 7.9, with the required kernel parameters applied

$ ./enroot-check_3.4.0_x86_64.run  --verify
Kernel version:

Linux version 3.10.0-1160.53.1.el7.x86_64 (mockbuild@kbuilder.bsys.centos.org) (gcc version 4.8.5 20150623 (Red Hat 4.8.5-44) (GCC) ) #1 SMP Fri Jan 14 13:59:45 UTC 2022

Kernel configuration:

CONFIG_NAMESPACES                 : OK
CONFIG_USER_NS                    : OK
CONFIG_SECCOMP_FILTER             : OK
CONFIG_OVERLAY_FS                 : OK (module)
CONFIG_X86_VSYSCALL_EMULATION     : KO (required if glibc <= 2.13)
CONFIG_VSYSCALL_EMULATE           : KO (required if glibc <= 2.13)
CONFIG_VSYSCALL_NATIVE            : KO (required if glibc <= 2.13)

Kernel command line:

namespace.unpriv_enable=1         : OK
user_namespace.enable=1           : OK
vsyscall=native                   : KO (required if glibc <= 2.13)
vsyscall=emulate                  : OK

Kernel parameters:

user.max_user_namespaces          : OK
user.max_mnt_namespaces           : OK

Extra packages:

nvidia-container-cli              : OK`

glibc is version 2.17 so I assume the "KO" fields are not an issue

$ ldd --version
ldd (GNU libc) 2.17
Copyright (C) 2012 Free Software Foundation, Inc.
This is free software; see the source for copying conditions.  There is NO
warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.
Written by Roland McGrath and Ulrich Drepper.

The following works fine

enroot import docker://alpine
enroot create alpine.sqsh
enroot start alpine
enroot remove alpine

But starting the container directly from the sqsh file fails:

$ enroot start alpine.sqsh
enroot-mount: failed to create file: /run/user/10001/enroot/overlay/etc/resolv.conf: No data available

I am using a completely empty /etc/enroot/enroot.conf file There is enough free disk space, and it happens independent of permissions (same for root user).

strace in the enroot-mount commands shows

...
mkdir("/run/user/10001/enroot/overlay/etc", 0755) = -1 EEXIST (File exists)
mknod("/run/user/10001/enroot/overlay/etc/resolv.conf", S_IFREG|000) = -1 ENODATA (No data available)
3XX0 commented 1 year ago

This is most likely due to fuse-overlayfs, you can try several versions of it see if the issue goes away.

Also make sure it works if you create a container first

ltalirz commented 1 year ago

Thanks a lot for the hint!

Indeed, a drop-in replacement of the fuse-overlayfs binary version 0.7.2 with version 1.10 fixed this issue.

Original version on CentOS 7.9

$ fuse-overlayfs --version
fuse-overlayfs: version 0.7.2
FUSE library version 3.6.1
using FUSE kernel interface version 7.29

New version

$ fuse-overlayfs --version
fuse-overlayfs: version 1.10
FUSE library version 3.10.5
using FUSE kernel interface version 7.31
ltalirz commented 1 year ago

@3XX0 after fixing this issue, I ran into other issues (depending on the image, enroot start hanging, other no data errors, ...)

Do I understand that these problem are actually with the Linux kernel version from above

Linux version 3.10.0-1160.53.1.el7.x86_64

that does not support enroot start for unprivileged users? (for root, enroot start works fine)

Would it make sense to add a check to the enroot-check.run script that checks the kernel version and warns users about the fact that enroot start will not work for unprivileged users?

While the requirement is mentioned here it is not mentioned in the requirements - would probably be useful to add a sentence

Linux Kernel >= 4.18 for enroot start in user space

there