Nvidia driver 560.35.03 cannot access gpu

valandyr commented 1 month ago

Description

firejail cannot access nvidia gpu. driver is functional outside of sandbox.

Steps to Reproduce

firejail --noprofile --private nvidia-smi

Expected behavior

Show GPU information/status.

Actual behavior

Failed to initialize NVML: GPU access blocked by the operating system

Behavior without a profile

Failed to initialize NVML: GPU access blocked by the operating system

Additional context

Last working driver is version 555.58.02

Environment

Linux archlinux 6.11.3-arch1-1 Arch Linux

firejail version 0.9.72 Compile time support:

always force nonewprivs support is disabled
AppArmor support is enabled
AppImage support is enabled
chroot support is enabled
D-BUS proxy support is enabled
file transfer support is enabled
firetunnel support is disabled
IDS support is disabled
networking support is enabled
output logging is enabled
overlayfs support is disabled
private-home support is enabled
private-cache and tmpfs as user enabled
SELinux support is disabled
user namespace support is enabled
X11 sandboxing support is enabled

Checklist

[x] The issues is caused by firejail (i.e. running the program by path (e.g. /usr/bin/vlc) "fixes" it).
[x] I can reproduce the issue without custom modifications (e.g. globals.local).
[ ] The program has a profile. (If not, request one in https://github.com/netblue30/firejail/issues/1139)
[ ] The profile (and redirect profile if exists) hasn't already been fixed upstream.
[x] I have performed a short search for similar issues (to avoid opening a duplicate).
- [ ] I'm aware of browser-allow-drm yes/browser-disable-u2f no in firejail.config to allow DRM/U2F in browsers.
[ ] I used --profile=PROFILENAME to set the right profile. (Only relevant for AppImages)

Log

Output of LC_ALL=C firejail /path/to/program

``` LC_ALL=C firejail --noprofile nvidia-smi Parent pid 11709, child pid 11710 Child process initialized in 8.91 ms Failed to initialize NVML: GPU access blocked by the operating system Parent is shutting down, bye... ```

Output of LC_ALL=C firejail --debug /path/to/program

``` LC_ALL=C firejail --noprofile --debug nvidia-smi Building quoted command line: 'nvidia-smi' Command name #nvidia-smi# DISPLAY=:1 parsed as 1 Using the local network stack Parent pid 11777, child pid 11778 Initializing child process Host network configured PID namespace installed Mounting tmpfs on /run/firejail/mnt directory Creating empty /run/firejail/mnt/seccomp directory Creating empty /run/firejail/mnt/seccomp/seccomp.protocol file Creating empty /run/firejail/mnt/seccomp/seccomp.postexec file Creating empty /run/firejail/mnt/seccomp/seccomp.postexec32 file Mounting /proc filesystem representing the PID namespace Basic read-only filesystem: Mounting read-only /etc 746 484 259:2 /etc /etc ro,relatime master:1 - ext4 /dev/nvme0n1p2 rw mountid=746 fsname=/etc dir=/etc fstype=ext4 Mounting noexec /etc 747 746 259:2 /etc /etc ro,nosuid,nodev,noexec,relatime master:1 - ext4 /dev/nvme0n1p2 rw mountid=747 fsname=/etc dir=/etc fstype=ext4 Mounting read-only /var 748 484 259:2 /var /var ro,relatime master:1 - ext4 /dev/nvme0n1p2 rw mountid=748 fsname=/var dir=/var fstype=ext4 Mounting noexec /var 749 748 259:2 /var /var ro,nosuid,nodev,noexec,relatime master:1 - ext4 /dev/nvme0n1p2 rw mountid=749 fsname=/var dir=/var fstype=ext4 Mounting read-only /usr 750 484 259:2 /usr /usr ro,relatime master:1 - ext4 /dev/nvme0n1p2 rw mountid=750 fsname=/usr dir=/usr fstype=ext4 Mounting tmpfs on /var/lock Mounting tmpfs on /var/tmp Mounting tmpfs on /var/log Create the new utmp file Mount the new utmp file Cleaning /home directory Cleaning /run/user directory Sanitizing /etc/passwd, UID_MIN 1000 Sanitizing /etc/group, GID_MIN 1000 Disable /run/firejail/sandbox Disable /run/firejail/network Disable /run/firejail/bandwidth Disable /run/firejail/name Disable /run/firejail/profile Disable /run/firejail/x11 blacklist /run/firejail/dbus Mounting read-only /proc/sys Remounting /sys directory Disable /sys/firmware Disable /sys/hypervisor Disable /sys/power Disable /sys/kernel/debug Disable /sys/kernel/vmcoreinfo Disable /proc/sys/fs/binfmt_misc Disable /proc/sys/kernel/core_pattern Disable /proc/sys/kernel/modprobe Disable /proc/sysrq-trigger Disable /proc/sys/vm/panic_on_oom Disable /proc/irq Disable /proc/bus Disable /proc/timer_list Disable /proc/kcore Disable /proc/kallsyms Disable /usr/lib/modules/6.11.3-arch1-1/build (requested /usr/src/linux) Disable /usr/lib/modules (requested /lib/modules) Disable /boot Disable /dev/port Disable /run/user/1000/gnupg Disable /run/user/1000/systemd Disable /dev/kmsg Disable /proc/kmsg Disable /sys/fs Disable /sys/module Current directory: /home/constance DISPLAY=:1 parsed as 1 Masking all X11 sockets except /tmp/.X11-unix/X1 Mounting read-only /run/firejail/mnt/seccomp 791 743 0:74 /seccomp /run/firejail/mnt/seccomp ro,nosuid - tmpfs tmpfs rw,mode=755,inode64 mountid=791 fsname=/seccomp dir=/run/firejail/mnt/seccomp fstype=tmpfs Seccomp directory: ls /run/firejail/mnt/seccomp drwxr-xr-x root root 120 . drwxr-xr-x root root 160 .. -rw-r--r-- constance constance 640 seccomp -rw-r--r-- constance constance 432 seccomp.32 -rw-r--r-- constance constance 0 seccomp.postexec -rw-r--r-- constance constance 0 seccomp.postexec32 No active seccomp files Drop privileges: pid 1, uid 1000, gid 1000, force_nogroups 0 Closing non-standard file descriptors Starting application LD_PRELOAD=(null) execvp argument 0: nvidia-smi Child process initialized in 13.21 ms Searching $PATH for nvidia-smi trying #/usr/local/sbin/nvidia-smi# trying #/usr/local/bin/nvidia-smi# trying #/usr/bin/nvidia-smi# Failed to initialize NVML: GPU access blocked by the operating system monitoring pid 2 Sandbox monitor: waitpid 2 retval 2 status 4352 Parent is shutting down, bye... ```

pjhfggij commented 1 month ago

UPDATE this suggestion solved this for me not sure why arch being a rolling distro still hasn't picked up these changes and everyone needs to look this up on individual basis add to corresponding profiles:

noblacklist /sys/module
whitelist /sys/module/nvidia*
read-only /sys/module/nvidia*

I've tried previously different combinations of these lines, but somehow they didn't do the trick for some reason - not sure why:

whitelist /sys/module/nvidia*
noblacklist /sys/module/nvidia*
ignore read-only /sys/module/nvidia*

I can reproduce this (Driver Version: 560.35.03, Arch) also I first noticed issues with nvidia card (optmius laptop) in March this year, when trying to run wine under firejail, it wasn't able to access nvidia card, but it worked fine without firejail I believe that it previously worked Tried adding this to wine.local, but to no effect:

noblacklist /dev/nvidia*
noblacklist /usr/lib/nvidia*
noblacklist /usr/lib/x86_64-linux-gnu/libGL.so*
noblacklist /usr/lib/x86_64-linux-gnu/libvulkan.so*

kmk3 commented 1 month ago

@pjhfggij on Oct 18:

UPDATE this suggestion solved this for me not sure why arch being a rolling distro still hasn't picked up these changes and everyone needs to look this up on individual basis add to corresponding profiles:

The changes from #6387 are only in the development version.

noblacklist /sys/module
whitelist /sys/module/nvidia*
read-only /sys/module/nvidia*

@valandyr

Does it work with the above lines?

Does it work with firejail-git (AUR)?

I've tried previously different combinations of these lines, but somehow they didn't do the trick for some reason - not sure why:
whitelist /sys/module/nvidia*
noblacklist /sys/module/nvidia*
ignore read-only /sys/module/nvidia*

A noblacklist line should match an existing blacklist line.

See blacklist /sys/module in fs.c.

rusty-snake commented 1 month ago

Behavior without a profile

Try firejail --profile=noprofile /usr/bin/nvidia-smi. If this fails, someone needs to strace into it.

valandyr commented 1 month ago

@rusty-snake firejail --profile=noprofile /usr/bin/nvidia-smi works! How does using a profile works and not using one fails?

In this context --noprofile --private=/path/to/dir --net=none --dbus-system=none --dbus-user=none --private-tmp --nonewprivs If i replace --noprofile with --profile=noprofile can i expect the same behavior?

@kmk3 firejail --noprofile --whitelist=/sys/module/nvidia* --read-only=/sys/module/nvidia* /usr/bin/nvidia-smi Failed to initialize NVML: GPU access blocked by the operating system Doesn't work.

rusty-snake commented 1 month ago

How does using a profile works and not using one fails?

--noprofile does not use a profile / an empty profile. noprofile.profile is a profile that does nothing beyond lifting (some) restrictions remaining with --noprofile (the most features is opt-in but a few are opt-out).

firejail --profile=noprofile /usr/bin/nvidia-smi works!

Play with the commands in noprofile.profile to findout which are required.

valandyr commented 1 month ago

(the most features is opt-in but a few are opt-out).

I assumed that --noprofile would disable everything, thanks for explaining.

Play with the commands in noprofile.profile to findout which are required.

firejail --noblacklist=/sys/module nvidia-smi works.

firejail --noblacklist=/sys/module/nvidia* nvidia-smi doesn't work. Probably additional modules are necessary with driver version 560.

Feel free to close this issue whenever.

rusty-snake commented 1 month ago

firejail --noblacklist=/sys/module nvidia-smi works. firejail --noblacklist=/sys/module/nvidia* nvidia-smi doesn't work.

As @kmk3 said: "A noblacklist line should [must] match an existing blacklist line."

Feel free to close this issue whenever.

"The changes from https://github.com/netblue30/firejail/pull/6387 are only in the development version."

I leave it open for now (~1day). Anybody else, fell free to close.

Can anybody with firejail from git-master and a nvidia system check that firejail --profile=noprofile ls /sys/module actually outputs the same as ls /sys/module. If it outputs nothing (i.e. empty directory), we have a "bug" in noprofile.profile.

kmk3 commented 1 month ago

Duplicate of #6372

netblue30 / firejail