Problems with private mode on Gentoo hardened

[Kalle72]

Hello,

I am on gentoo hardened and have one problem and one question ;-)

First, the question: Starting firejail (e.g. firejail firefox) says among others "Warning: cannot disable /sys/hypervisor directory", but it seems to start properly. I looked as root for /sys/hypervisor, but this directory does not exist. So is this a problem or can I ignore it?

Second, the problem: If I start the private mode (e.g. firejail --private firefox) it does not start and give me the following failure: "Error mounting home directory:fs_private(236): No such file or directory Error: cannot establish communication with the parent, exiting..." On the other hand "firejail --private-home.mozilla firefox" works. Any idea what the problem could be?

Thanks in advance and best regards Kalle

PS: Because I am on Gentoo and use a self-configured kernel it might be possible that a kernel feature is missing.

[netblue30]

I've fixed the "/sys/hypervisor directory" problem. Some module is not compiled in the kernel, I shouldn't put out a warning.

Let's take a look at your kernel. For namespaces I have (kernel 3.18):

CONFIG_NAMESPACES=y
CONFIG_UTS_NS=y
CONFIG_IPC_NS=y
CONFIG_USER_NS=y
CONFIG_PID_NS=y
CONFIG_NET_NS=y

I think this is the only thing required.

"Error mounting home directory:fs_private(236)" - This is very strange, it fails to mount a tmpfs on /home or /root directories. Do you have by any chance /root, /home or /home/username mounted on a separate partition?

Thanks for the bug!

[Kalle72]

Thanks for your answer. Yes /home is mounted on a separate partition. All the kernel-feature you mentioned are activated.

I will try it with a kernel where I deactivate all GrSec-features.

[Kalle72]

Tested it: Disabling all GrSec-features does not solve the problem. Everything still the same. Next I will test what happens if I move my /home folder to the root-partition /

[Kalle72]

Moved my home-folder to my root-partition, but does not help. So I think it is not related to /home on a separate partition.

[Kalle72]

I dont know if this is of interest, but I found out that "firejail --private-home=.mozilla firefox" works, as written above, against what "firejail --private-home=.mozilla --whitelist=/home/kalle/Downloads firefox" leads to the same error as "firejail --private firefox"

[netblue30]

I did some tests here, it doesn't matter where the partition is. Can you please run it with --debug flag, and dump the result here:

$ firejail --debug --private firefox

[Kalle72]

Thanks for your time!!! First: I saw that my version of firejail (0.9.32) is outdated and made an upgrade to (0.9.34). Gentoo-overlays are not up to date here. However, the result is that the problem occurs now also when I start firefox with firejails default settings (firejail firefox). However: Thunderbird still works with the default settings, but not in private mode (same error as firefox).

I will try it again with a vanilla-kernel (maybe some hardening features of GrSec are activen even if GrSec is disabled)

Second: The output you requested (with firejail 0.9.34)

karl@nuth ~ $ firejail --debug firefox Command name #firefox# Found firefox profile in /etc/firejail directory Reading profile /etc/firejail/firefox.profile Checking filename ${HOME}/.mozilla Reading profile /etc/firejail/disable-mgmt.inc Checking filename /sbin Checking filename /usr/sbin Checking filename ${PATH}/umount Checking filename ${PATH}/mount Checking filename ${PATH}/fusermount Checking filename ${PATH}/su Checking filename ${PATH}/sudo Checking filename ${PATH}/xinput Checking filename ${PATH}/evtest Checking filename ${PATH}/xev Checking filename ${PATH}/strace Checking filename /etc/firejail Checking filename ${HOME}/.config/firejail Reading profile /etc/firejail/disable-secret.inc Checking filename ${HOME}/.ssh Checking filename ${HOME}/.gnome2private Checking filename ${HOME}/.gnome2/keyrings Checking filename ${HOME}/kde4/share/apps/kwallet Checking filename ${HOME}/kde/share/apps/kwallet Checking filename ${HOME}/.netrc Checking filename ${HOME}/.gnupg Checking filename ${HOME}/.local/share/recently-used.xbel Checking filename ${HOME}/.kdb Checking filename ${HOME}/_.key Reading profile /etc/firejail/disable-common.inc Checking filename ${HOME}/.history Checking filename ${HOME}/.__history Checking filename ${HOME}/.adobe Checking filename ${HOME}/.macromedia Checking filename ${HOME}/.mozilla Checking filename ${HOME}/.icedove Checking filename ${HOME}/.thunderbird Checking filename ${HOME}/.sylpheed-2.0 Checking filename ${HOME}/.config/midori Checking filename ${HOME}/.config/opera Checking filename ${HOME}/.config/chromium Checking filename ${HOME}/.config/google-chrome Checking filename ${HOME}/.filezilla Checking filename ${HOME}/.config/filezilla Checking filename ${HOME}/.local/share/systemd Checking filename ${HOME}/.config/hexchat Checking filename ${HOME}/.mcabber Checking filename ${HOME}/.purple Checking filename ${HOME}/.config/psi+ Checking filename ${HOME}/.retroshare Checking filename ${HOME}/.weechat Checking filename ${HOME}/.config/xchat Checking filename ${HOME}/._coin Checking filename ${HOME}/.electrum Checking filename ${HOME}/wallet.dat Checking filename ${HOME}/.remmina Checking filename ${HOME}/.tconn Checking filename ${HOME}/.FBReader Checking filename ${HOME}/.xinitrc Checking filename ${HOME}/.xprofile Checking filename ${HOME}/.config/autostart Checking filename /etc/xdg/autostart Checking filename ${HOME}/.kde4/Autostart Checking filename ${HOME}/.kde4/share/autostart Checking filename ${HOME}/.kde/Autostart Checking filename ${HOME}/.config/plasma-workspace/shutdown Checking filename ${HOME}/.config/plasma-workspace/env Checking filename ${HOME}/.config/lxsession/LXDE/autostart Checking filename ${HOME}/.fluxbox/startup Checking filename ${HOME}/.config/openbox/autostart Checking filename ${HOME}/.config/openbox/environment Checking filename ${HOME}/.VirtualBox Checking filename ${HOME}/VirtualBox VMs Checking filename ${HOME}/.config/VirtualBox Checking filename ${HOME}/.subversion Checking filename ${HOME}/.gitconfig Checking filename ${HOME}/.git-credential-cache Checking filename /var/spool/cron Checking filename /var/spool/anacron Checking filename /var/run/acpid.socket Checking filename /var/run/minissdpd.sock Checking filename /var/run/rpcbind.sock Checking filename /var/run/mysqld/mysqld.sock Checking filename /var/run/mysql/mysqld.sock Checking filename /var/lib/mysqld/mysql.sock Checking filename /var/lib/mysql/mysql.sock Checking filename /var/run/docker.sock Checking filename /etc/cron. Checking filename /etc/profile.d Checking filename /etc/rc.local Checking filename /etc/anacrontab Checking filename ${HOME}/.xinitrc Checking filename ${HOME}/.xserverrc Checking filename ${HOME}/.profile Checking filename ${HOME}/.bash_login Checking filename ${HOME}/.bashrc Checking filename ${HOME}/.bash_profile Checking filename ${HOME}/.bash_logout Checking filename ${HOME}/.zshrc Checking filename ${HOME}/.zlogin Checking filename ${HOME}/.zprofile Checking filename ${HOME}/.zlogout Checking filename ${HOME}/.zsh_files Checking filename ${HOME}/.tcshrc Checking filename ${HOME}/.cshrc Checking filename ${HOME}/.csh_files Checking filename ${HOME}/.mailcap Checking filename ${HOME}/.exrc Checking filename ${HOME}/.vimrc Checking filename ${HOME}/.vim Checking filename ${HOME}/.emacs Checking filename ${HOME}/.tmux.conf Checking filename ${HOME}/.iscreenrc Checking filename ${HOME}/.muttrc Checking filename ${HOME}/.xmonad Checking filename ${HOME}/bin Reading profile /etc/firejail/disable-devel.inc Checking filename /usr/include Checking filename /usr/bin/gcc Checking filename /usr/bin/cpp Checking filename /usr/bin/c9 Checking filename /usr/bin/c8 Checking filename /usr/bin/c++ Checking filename /usr/bin/ld Checking filename /usr/bin/valgrind Checking filename /usr/lib/valgrind Checking filename /usr/bin/perl Checking filename /usr/bin/cpan Checking filename /usr/share/perl Checking filename /usr/lib/perl Checking filename /usr/bin/php Checking filename /usr/share/php Checking filename /usr/lib/php Checking filename /usr/bin/ruby Checking filename /usr/lib/ruby Checking filename ~/.mozilla Checking filename ~/Downloads Checking filename ~/dwhelper Checking filename ~/.zotero Checking filename ~/.lastpass Checking filename ~/.gtkrc-2.0 Checking filename ~/.vimperatorrc Checking filename ~/.vimperator Checking filename ~/.pentadactylrc Checking filename ~/.pentadactyl Checking filename ~/.fonts Checking filename ~/.fonts.d Checking filename ~/.fontconfig Checking filename ~/.fonts.conf Checking filename ~/.fonts.conf.d Using the local network stack Parent pid 2355, child pid 2356 Initializing child process PID namespace installed Mounting tmpfs on /tmp/firejail/mnt directory Mounting read-only /bin, /sbin, /lib, /lib64, /usr, /etc, /var Mounting tmpfs on /var/lock Mounting tmpfs on /var/tmp Mounting tmpfs on /var/log Mounting tmpfs on /var/lib/dhcp Create the new utmp file Mount the new utmp file Replaced whitelist path: whitelist /home/karl/.mozilla Removed whitelist path: whitelist ~/Downloads Removed whitelist path: whitelist ~/dwhelper Removed whitelist path: whitelist ~/.zotero Removed whitelist path: whitelist ~/.lastpass Removed whitelist path: whitelist ~/.gtkrc-2.0 Removed whitelist path: whitelist ~/.vimperatorrc Removed whitelist path: whitelist ~/.vimperator Removed whitelist path: whitelist ~/.pentadactylrc Removed whitelist path: whitelist ~/.pentadactyl Removed whitelist path: whitelist ~/.fonts Removed whitelist path: whitelist ~/.fonts.d Removed whitelist path: whitelist ~/.fontconfig Removed whitelist path: whitelist ~/.fonts.conf Removed whitelist path: whitelist ~/.fonts.conf.d Mounting a new /home directory Mounting a new /root directory Error mounting home directory:fs_private(230): No such file or directory Error: cannot establish communication with the parent, exiting...

[Kalle72]

Tested it with vanilla-sources not patched with GrSecurity at all. Does not work too! So three scenarios are possible: 1) The hardened Toolchain is the problem 2) Some things inside the kernel are missing 3) Some additional program is needed Next I will test it on a machine without hardened toolchain to exclude this possibility from the list.

[Kalle72]

Installed an Ubuntu-Kernel on my machine, because with Ubuntu in a VirtualBox it works. Result: Does not work. So I think the problem is not related to my kernel-config

Additionally I tested it on a one of my Gentoo-machines without hardened toolchain. Result: Does not work too.

So can it be that there is a problem with the file-system I use (xfs) or that some additional programs are needed which are installed by default on other distros?

The mount of the new /home partition seems to be the problem. How does this mount work exactly?

[netblue30]

It tries to mount a tmpfs on top of /root directory and fails.

Can you try to run the latest version on master branch? You would go an get the latest zip archive - on the main page or from here https://github.com/netblue30/firejail/archive/master.zip

You unzip it (unzip firejail-master.zip), go into the directory (cd firejail-master), configure and compile (./configure --prefix=/usr && make && sudo make install). The code creating the problem is in src/fs_home.c at line 236. It looks like this:

    // mask /root
    if (arg_debug)
        printf("Mounting a new /root directory\n");
    if (mount("tmpfs", "/root", "tmpfs", MS_NOSUID | MS_NODEV | MS_STRICTATIME | MS_REC,  "mode=700,gid=0") < 0)
        errExit("mounting root directory");
    fs_logger("mount tmpfs on /root");

If we just comment out this code (add a "//" at the beginning of every line), would this work?

[Kalle72]

Unfortunately it does not work. But the error changed ;-) The output is now (normal and debug mode):

karl@nuth ~ $ firejail firefox Reading profile /etc/firejail/firefox.profile Reading profile /etc/firejail/disable-mgmt.inc Reading profile /etc/firejail/disable-secret.inc Reading profile /etc/firejail/disable-common.inc Reading profile /etc/firejail/disable-devel.inc Reading profile /etc/firejail/whitelist-common.inc Parent pid 3542, child pid 3543 Error: file /home/karl is not in user home directory, exiting... Error: cannot establish communication with the parent, exiting... karl@nuth ~ $ firejail --debug firefox Command name #firefox# Found firefox profile in /etc/firejail directory Reading profile /etc/firejail/firefox.profile Reading profile /etc/firejail/disable-mgmt.inc Reading profile /etc/firejail/disable-secret.inc Reading profile /etc/firejail/disable-common.inc Reading profile /etc/firejail/disable-devel.inc Reading profile /etc/firejail/whitelist-common.inc Using the local network stack Parent pid 3633, child pid 3634 Initializing child process PID namespace installed Mounting tmpfs on /run/firejail/mnt directory Mounting read-only /bin, /sbin, /lib, /lib32, /lib64, /usr, /etc, /var Mounting tmpfs on /var/lock Mounting tmpfs on /var/tmp Mounting tmpfs on /var/log Mounting tmpfs on /var/lib/dhcp Create the new utmp file Mount the new utmp file Cleaning /home directory Sanitizing /etc/passwd Sanitizing /etc/group Disable /etc/firejail Downloads directory resolved as "" Replaced whitelist path: whitelist /home/karl Replaced whitelist path: whitelist /home/karl/.mozilla Replaced whitelist path: whitelist /home/karl/.cache/mozilla/firefox Removed whitelist path: whitelist ~/dwhelper expanded: /home/karl/dwhelper real path: (null) realpath: No such file or directory Removed whitelist path: whitelist ~/.zotero expanded: /home/karl/.zotero real path: (null) realpath: No such file or directory Removed whitelist path: whitelist ~/.lastpass expanded: /home/karl/.lastpass real path: (null) realpath: No such file or directory Removed whitelist path: whitelist ~/.vimperatorrc expanded: /home/karl/.vimperatorrc real path: (null) realpath: No such file or directory Removed whitelist path: whitelist ~/.vimperator expanded: /home/karl/.vimperator real path: (null) realpath: No such file or directory Removed whitelist path: whitelist ~/.pentadactylrc expanded: /home/karl/.pentadactylrc real path: (null) realpath: No such file or directory Removed whitelist path: whitelist ~/.pentadactyl expanded: /home/karl/.pentadactyl real path: (null) realpath: No such file or directory Removed whitelist path: whitelist ~/.config/gnome-mplayer expanded: /home/karl/.config/gnome-mplayer real path: (null) realpath: No such file or directory Removed whitelist path: whitelist ~/.cache/gnome-mplayer/plugin expanded: /home/karl/.cache/gnome-mplayer/plugin real path: (null) realpath: No such file or directory Replaced whitelist path: whitelist /home/karl/.config/mimeapps.list Removed whitelist path: whitelist ~/.icons expanded: /home/karl/.icons real path: (null) realpath: No such file or directory Replaced whitelist path: whitelist /home/karl/.config/user-dirs.dirs Removed whitelist path: whitelist ~/.fonts expanded: /home/karl/.fonts real path: (null) realpath: No such file or directory Removed whitelist path: whitelist ~/.fonts.d expanded: /home/karl/.fonts.d real path: (null) realpath: No such file or directory Replaced whitelist path: whitelist /home/karl/.fontconfig Removed whitelist path: whitelist ~/.fonts.conf expanded: /home/karl/.fonts.conf real path: (null) realpath: No such file or directory Removed whitelist path: whitelist ~/.fonts.conf.d expanded: /home/karl/.fonts.conf.d real path: (null) realpath: No such file or directory Removed whitelist path: whitelist ~/.gtkrc expanded: /home/karl/.gtkrc real path: (null) realpath: No such file or directory Replaced whitelist path: whitelist /home/karl/.gtkrc-2.0 Replaced whitelist path: whitelist /home/karl/.config/gtk-3.0 Removed whitelist path: whitelist ~/.themes expanded: /home/karl/.themes real path: (null) realpath: No such file or directory Mounting a new /home directory Create a new user directory Error: file /home/karl is not in user home directory, exiting... Error: cannot establish communication with the parent, exiting... karl@nuth ~ $

[Kalle72]

By the way. Is the new /root or the new /home the problem or both?

[netblue30]

Only /root is the problem, the message was wrong, now I have it fixed. It tries to mount a temporary filesystem (tmpfs) on top of /root directory, in order to mask all the files there. Does this work form command line on your system?

(as root)
# mount -t tmpfs tmpfs /root
# grep root /etc/mtab
[...]
tmpfs /root tmpfs rw,relatime 0 0

Going to "file /home/karl is not in user home directory, exiting". This is a very ugly bug on my side, thank you for bringing it up!

Downloads directory resolved as ""
Replaced whitelist path: whitelist /home/karl
[...]
Error: file /home/karl is not in user home directory, exiting...

Workaround: create a Downloads directory in your home directory ($ mkdir ~/Downloads). Also, can you please print here the contents of ~/.config/user-dirs.dirs. Thanks.

[Kalle72]

Many thanks for your efforts!!! The /root problem is solved. In detail: My root folder was mounted to /home/root and /root was a symlink to it. Therefore "mount -t tmpfs tmpfs /root" mounted the tmpfs to /home/root with the result in not finding it after, because mtab showed it as mounted in /home/root and not in /root.

My user-dir.dirs was (no other entries) XDG_DESKTOP_DIR="$HOME/Desktop" XDG_DOWNLOAD_DIR="$HOME/"

This is related to the fact that I deleted all visible directories XFCE created at first start. Until now I specified the download-location for every download separately in firefox. I decided now to switch to the following setup (Made it default in /etc/xdg/user-dir.default): XDG_DESKTOP_DIR="$HOME/Desktop" XDG_DOWNLOAD_DIR="$HOME/.Download" and created a link to .Download on the Desktop. Now .Download is my default download location (also in firefox). (Seems more secure then let firefox view the whole home-folder)

Many thanks again! Everything seems to work now ;-) Best regards Kalle

[Kalle72]

One last comment: If no download directory is specified in user-dir.dir at all or the download directory is set to the whole home directory like it was on my machine or the download directory is specified but not exisitent, firejail could do the following:

Set the Download directory to the default (~/Downloads) and create ~/Downloads. Then firejail could print a short error message how to change if one runs it in a terminal.

PS: If there are things to test related to GrSecurity (I think on https://github.com/netblue30/firejail/issues/141) then let me know.