/etc is unwritable on --chroot on debootstrap system

rayment commented 2 years ago

Description

Following the documentation at https://firejail.wordpress.com/documentation-2/basic-usage/ concerning the usage of firejail for creating an isolated debootstrap system, I am unable to create a user account and installing packages via. apt result in errors when attempting write to /etc.

Steps to Reproduce

(as root)

emerge -qv firejail
echo "force-nonewprivs yes" >> /etc/firejail/firejail.config
mkdir /jail
debootstrap --arch=amd64 stable /jail
LC_ALL=C TERM=xterm-color firejail --noprofile --chroot=/jail --shell=/bin/bash (note this already deviates from the documentation as the provided command will actually fail without setting --shell)
adduser foo

Expected behavior

$ adduser foo
Adding user `foo' ...
Adding new group `foo' (1000) ...
etc. etc.

Actual behavior

$ adduser foo
Adding user `foo' ...
Adding new group `foo' (1000) ...
groupadd: failure while writing changes to /etc/group
adduser: `/sbin/groupadd -g 1000 foo' returned error code 10. Exiting.

Additional context

I'm not sure if this is really a bug or merely a configuration error or a lack of concise documentation, but this occurs using the default firejail v0.9.68 as it comes on Gentoo with the only change in configuration being force-nonewprivs yes as suggested by the documentation.

Similar commands such as useradd or unrelated commands like calling apt are also failing:

$ useradd foo
useradd: failure while writing changes to /etc/passwd

$ apt install htop
...
ldconfig: Renaming of /etc/ld.so.cache~ to /etc/ld.so.cache failed: Device or resource busy
dkpg: error processing package libc-bin (--configure):
  installed libc-bin package post-installation script subprocess returned error exit status 1
Errors were encountered while processing:
  libc-bin
E: Sub-process /usr/bin/dpkg returned an error code (1)

Environment

Linux distribution and version (e.g. "Ubuntu 20.04" or "Arch Linux")

Linux home 5.15.41-gentoo-x86_64 #1 SMP Thu Jun 30 20:08:43 UTC 2022 x86_64 AMD Ryzen 7 PRO 5850U with Radeon Graphics AuthenticAMD GNU/Linux

Gentoo USE flags

X chroot dbusproxy file-transfer globalcfg network private-home userns -apparmor -contrib -test

Firejail version (firejail --version).

firejail version 0.9.68

Compile time support:
    - always force nonewprivs support is disabled
    - AppArmor support is disabled
    - AppImage support is enabled
    - chroot support is enabled
    - D-BUS proxy support is enabled
    - file transfer support is enabled
    - firetunnel support is disabled
    - networking support is enabled
    - output logging is enabled
    - overlayfs support is disabled
    - private-home support is enabled
    - private-cache and tmpfs as user enabled
    - SELinux support is disabled
    - user namespace support is enabled
    - X11 sandboxing support is enabled

Checklist

[x] The issues is caused by firejail (i.e. running the program by path (e.g. /usr/bin/vlc) "fixes" it).
- Yes, a regular chroot lets me interact with my Debian installation as normal, but that defeats the purpose.
[x] I can reproduce the issue without custom modifications (e.g. globals.local).
[ ] The program has a profile. (If not, request one in https://github.com/netblue30/firejail/issues/1139)
- --noprofile as per the docs.
[ ] The profile (and redirect profile if exists) hasn't already been fixed upstream.
[x] I have performed a short search for similar issues (to avoid opening a duplicate).
- [x] I'm aware of browser-allow-drm yes/browser-disable-u2f no in firejail.config to allow DRM/U2F in browsers.
[ ] I used --profile=PROFILENAME to set the right profile. (Only relevant for AppImages)

Log

Output of LC_ALL=C firejail --debug /path/to/program

``` $ LC_ALL=C TERM=xterm-color firejail --noprofile --chroot=/jail --shell=/bin/bash --debug Command name #/bin/bash# Enabling IPC namespace Using the local network stack Command name #/bin/bash# Enabling IPC namespace Using the local network stack Initializing child process PID namespace installed Mounting tmpfs on /run/firejail/mnt directory Creating empty /run/firejail/mnt/seccomp directory Creating empty /run/firejail/mnt/seccomp/seccomp.protocol file Creating empty /run/firejail/mnt/seccomp/seccomp.postexec file Creating empty /run/firejail/mnt/seccomp/seccomp.postexec32 file Mounting /dev on chroot /dev Updating chroot /etc/resolv.conf Chrooting into /jail Mounting /proc filesystem representing the PID namespace Mounting tmpfs on /var/lock Mounting tmpfs on /var/tmp Mounting tmpfs on /var/log Create the new utmp file Mount the new utmp file blacklist /run/firejail/dbus Mounting read-only /proc/sys Remounting /sys directory Disable /sys/firmware Disable /sys/power Disable /sys/kernel/debug Disable /sys/kernel/vmcoreinfo Disable /proc/sys/fs/binfmt_misc Disable /proc/sys/kernel/core_pattern Disable /proc/sys/kernel/modprobe Disable /proc/sysrq-trigger Disable /proc/sys/vm/panic_on_oom Disable /proc/irq Disable /proc/bus Disable /proc/timer_list Disable /proc/kcore Disable /proc/kallsyms Disable /boot Disable /dev/port Disable /mnt Disable /media Disable /run/mount Disable /sys/fs Disable /sys/module rebuilding /etc directory Creating empty /run/firejail/mnt/dns-etc/rc2.d directory Creating empty /run/firejail/mnt/dns-etc/xattr.conf file Creating empty /run/firejail/mnt/dns-etc/selinux directory Creating empty /run/firejail/mnt/dns-etc/fstab file Creating empty /run/firejail/mnt/dns-etc/group- file Creating empty /run/firejail/mnt/dns-etc/apt directory Creating empty /run/firejail/mnt/dns-etc/ca-certificates.conf file Creating empty /run/firejail/mnt/dns-etc/resolv.conf file Creating empty /run/firejail/mnt/dns-etc/rcS.d directory Creating empty /run/firejail/mnt/dns-etc/kernel directory Creating empty /run/firejail/mnt/dns-etc/timezone file Creating empty /run/firejail/mnt/dns-etc/passwd file Creating empty /run/firejail/mnt/dns-etc/ld.so.cache file Creating empty /run/firejail/mnt/dns-etc/ld.so.conf file Creating empty /run/firejail/mnt/dns-etc/rc0.d directory Creating empty /run/firejail/mnt/dns-etc/host.conf file Creating empty /run/firejail/mnt/dns-etc/gshadow file Creating empty /run/firejail/mnt/dns-etc/adduser.conf file Creating empty /run/firejail/mnt/dns-etc/systemd directory Creating empty /run/firejail/mnt/dns-etc/nsswitch.conf file Creating empty /run/firejail/mnt/dns-etc/ld.so.conf.d directory Creating empty /run/firejail/mnt/dns-etc/debian_version file Creating empty /run/firejail/mnt/dns-etc/subgid file Creating empty /run/firejail/mnt/dns-etc/cron.d directory Creating empty /run/firejail/mnt/dns-etc/rc6.d directory Creating empty /run/firejail/mnt/dns-etc/mke2fs.conf file Creating empty /run/firejail/mnt/dns-etc/default directory Creating empty /run/firejail/mnt/dns-etc/deluser.conf file Creating empty /run/firejail/mnt/dns-etc/dpkg directory Creating empty /run/firejail/mnt/dns-etc/pam.d directory Creating empty /run/firejail/mnt/dns-etc/subuid file Creating empty /run/firejail/mnt/dns-etc/rc3.d directory Creating empty /run/firejail/mnt/dns-etc/issue.net file Creating empty /run/firejail/mnt/dns-etc/bash.bashrc file Creating empty /run/firejail/mnt/dns-etc/profile.d directory Creating empty /run/firejail/mnt/dns-etc/netconfig file Creating empty /run/firejail/mnt/dns-etc/rc5.d directory Creating empty /run/firejail/mnt/dns-etc/shells file Creating empty /run/firejail/mnt/dns-etc/ca-certificates directory Creating empty /run/firejail/mnt/dns-etc/.pwd.lock file Creating empty /run/firejail/mnt/dns-etc/update-motd.d directory Creating empty /run/firejail/mnt/dns-etc/shadow- file Creating empty /run/firejail/mnt/dns-etc/hostname file Creating empty /run/firejail/mnt/dns-etc/debconf.conf file Creating empty /run/firejail/mnt/dns-etc/passwd- file Creating empty /run/firejail/mnt/dns-etc/environment file Creating empty /run/firejail/mnt/dns-etc/logrotate.d directory Creating empty /run/firejail/mnt/dns-etc/e2scrub.conf file Creating empty /run/firejail/mnt/dns-etc/opt directory Creating empty /run/firejail/mnt/dns-etc/rc1.d directory Creating empty /run/firejail/mnt/dns-etc/libaudit.conf file Creating empty /run/firejail/mnt/dns-etc/ssl directory Creating empty /run/firejail/mnt/dns-etc/gai.conf file Creating empty /run/firejail/mnt/dns-etc/bindresvport.blacklist file Creating empty /run/firejail/mnt/dns-etc/cron.daily directory Creating empty /run/firejail/mnt/dns-etc/gss directory Creating empty /run/firejail/mnt/dns-etc/profile file Creating empty /run/firejail/mnt/dns-etc/motd file Creating empty /run/firejail/mnt/dns-etc/shadow file Creating empty /run/firejail/mnt/dns-etc/skel directory Creating empty /run/firejail/mnt/dns-etc/pam.conf file Creating empty /run/firejail/mnt/dns-etc/group file Creating empty /run/firejail/mnt/dns-etc/terminfo directory Creating empty /run/firejail/mnt/dns-etc/issue file Creating empty /run/firejail/mnt/dns-etc/security directory Creating empty /run/firejail/mnt/dns-etc/login.defs file Creating empty /run/firejail/mnt/dns-etc/init.d directory Creating empty /run/firejail/mnt/dns-etc/rc4.d directory Creating empty /run/firejail/mnt/dns-etc/alternatives directory Mount-bind /run/firejail/mnt/dns-etc on top of /etc Current directory: /root Mounting read-only /run/firejail/mnt/seccomp 279 109 0:50 /seccomp /run/firejail/mnt/seccomp ro,nosuid - tmpfs tmpfs rw,mode=755,inode64 mountid=279 fsname=/seccomp dir=/run/firejail/mnt/seccomp fstype=tmpfs Seccomp directory: ls /run/firejail/mnt/seccomp drwxr-xr-x root root 120 . drwxr-xr-x root root 180 .. -rw-r--r-- root root 568 seccomp -rw-r--r-- root root 432 seccomp.32 -rw-r--r-- root root 0 seccomp.postexec -rw-r--r-- root root 0 seccomp.postexec32 No active seccomp files NO_NEW_PRIVS set Drop privileges: pid 1, uid 0, gid 0, force_nogroups 0 No supplementary groups Closing non-standard file descriptors Starting application LD_PRELOAD=(null) Starting /bin/bash shell execvp argument 0: /bin/bash The new log directory is /proc/14520/root/var/log ```

rusty-snake commented 2 years ago

Can you try with --writable-etc.

note this already deviates from the documentation as the provided command will actually fail without setting --shell)

There were recent changes to --shell but it should be investigated what --chroot changes on it.

rayment commented 2 years ago

Can you try with --writable-etc.

Unfortunately it seems to change nothing, that is, exact same errors.

smitsohu commented 2 years ago

The reason is probably that nowadays Firejail creates lots of mount points in /etc, and kernel doesn't allow us to rename a mount point.

@rayment Just for me to clarify, do you use any of --dns or --ip=dhcp or --ip6=dhcp?

rusty-snake commented 2 years ago

The reason is probably that nowadays Firejail creates lots of mount points in /etc,

https://github.com/netblue30/firejail/issues/5010#issuecomment-1098700858

rayment commented 2 years ago

The reason is probably that nowadays Firejail creates lots of mount points in /etc, and kernel doesn't allow us to rename a mount point.

@rayment Just for me to clarify, do you use any of --dns or --ip=dhcp or --ip6=dhcp?

No I wasn't - only as shown in the bug report.

$ LC_ALL=C TERM=xterm-color firejail --noprofile --ip=dhcp --chroot=/jail --shell=/bin/bash
Error: No network device configured

$ LC_ALL=C TERM=xterm-color firejail --noprofile --ip6=dhcp --chroot=/jail --shell=/bin/bash
Error: No network device configured

$ LC_ALL=C TERM=xterm-color firejail --noprofile --dns=8.8.8.8 --chroot=/jail --shell=/bin/bash
&c &c

DNS server 8.8.8.8

&c &c
Child process initialized in 5.45 ms
# useradd foo
useradd: failed while writing changes to /etc/passwd

rayment commented 2 years ago

For what it's worth, I've tried the --writable-etc flag with --read-write on a combination of files and folders including /etc and /etc/passwd with no success.

smitsohu commented 2 years ago

If the conclusion from #5010 is to revert the offending commit, then that would obviously solve this issue as well.

Otherwise, would it make sense to add a hotfix in master?

chroot-hotfix.patch.txt

rayment commented 2 years ago

If the conclusion from #5010 is to revert the offending commit, then that would obviously solve this issue as well.

Otherwise, would it make sense to add a hotfix in master?

chroot-hotfix.patch.txt

I can confirm that this completely fixes my issue by using a custom ebuild on Gentoo with this patch compiled with 0.9.70.

While I wait for an update I'll use this solution, thank you very much.

netblue30 / firejail