search

mviereck / x11docker

Run GUI applications and desktops in docker and podman containers. Focus on security.
MIT License
5.61k stars 377 forks source link

Offer a way to provide preferred GPU in case of multi-GPU setups #483

Open awerlang opened 1 year ago

awerlang commented 1 year ago

Sample inxi output:

$ inxi -G
Graphics:
  Device-1: Intel HD Graphics 630 driver: i915 v: kernel
  Device-2: NVIDIA GP107 [GeForce GTX 1050 Ti] driver: nvidia v: 525.85.05
  Display: x11 server: X.Org v: 21.1.6 driver: X: loaded: modesetting,nvidia
    dri: iris gpu: i915 resolution: 1: 2560x1440~60Hz 2: 2560x1440~60Hz
  API: OpenGL v: 4.6 Mesa 22.3.4 renderer: Mesa Intel HD Graphics 630 (KBL
    GT2)

In this setup, NVIDIA is available but only Intel is used by Xorg. Some options:

Reproduction:

Execute x11docker with --gpu flag.

Observed behavior:

NVIDIA devices are probed, attempted to be installed.

Expected behavior:

Ignore NVIDIA even though NVIDIA proprietary drivers are installed.

awerlang commented 1 year ago

During some tests with sway, I found that containerized software can still query loaded kernel drivers. I ended up removing proprietary driver and blacklisting nouveau, but it could also work by masking /proc/driver/nvidia.

awerlang commented 1 year ago

I switched Xorg to leverage nvidia with proprietary drivers again (no xorg.conf):

$ inxi -G
Graphics:
  Device-1: Intel HD Graphics 630 driver: i915 v: kernel
  Device-2: NVIDIA GP107 [GeForce GTX 1050 Ti] driver: nvidia v: 525.85.05
  Device-3: Logitech HD Pro Webcam C920 type: USB
    driver: snd-usb-audio,uvcvideo
  Display: x11 server: X.Org v: 21.1.6 driver: X: loaded: modesetting,nvidia
    unloaded: fbdev,vesa dri: iris gpu: i915 resolution: 1: 2560x1440~60Hz
    2: 2560x1440~60Hz
  API: OpenGL v: 4.6.0 NVIDIA 525.85.05 renderer: NVIDIA GeForce GTX 1050
    Ti/PCIe/SSE2

I have commented out paths in x11docker which attempt to prepare nvidia stuff and ran with --security-opt mask=/proc/driver but it doesn't seem to be enough.

Base command line:

x11docker --backend=podman --xc --cap-default --newprivileges=no --init=systemd --home --network --clipboard --webcam --pulseaudio=host --gpu --xorg -- --security-opt mask=/proc/driver -- awerlang/citrix

--xpra-xwayland fails

(xpra:141): Gdk-ERROR **: 13:22:31.608: The program 'xpra' received an X Window System error.
This probably reflects a bug in the program.
The error was 'GLXBadContext'.
  (Details: serial 270 error_code 158 request_code 152 (GLX) minor_code 6)
  (Note to programmers: normally, X errors are reported asynchronously;
   that is, you will receive the error a while after causing it.
   To debug your program, run it with the GDK_SYNCHRONIZE environment
   variable to change this behavior. You can then get a meaningful
   backtrace from your debugger if you break on the gdk_x_error() function.)
Trace/breakpoint trap (core dumped)

xterm: fatal IO error 104 (Connection reset by peer) or KillClient on X server ":105"

--nxagent is able to launch xterm but firefox fails with a segmentation fault

--xorg fails

(EE) xf86OpenConsole: Cannot open virtual console 8 (Permission denied)

--hostdisplay fails

Invalid MIT-MAGIC-COOKIE-1 key
Authorization required, but no authorization protocol specified

In fact, only --nxagent works if I remove --gpu, other options fail to launch.

TLDR; I'd like to use i915 inside container independently of what Xorg's running.

mviereck commented 1 year ago

Sorry for my late response and short answer. I'll look later at your suggestions. As a general idea: You could run without --gpu, use option --weston-xwayland and share desired device files yourself. Example free of NVIDIA:

x11docker --weston-xwayland --share=/dev/dri --group-add=video --group-add=render [...]
awerlang commented 1 year ago

Hey, no worries! I'll try your suggestion and report. Also I am hopefully able to run Xorg on i915 again (troubleshooting freezes likely unrelated to GPUs but CPU overheating). I still have nvidia proprietary installed so I'll see if that interferes with anything.

mviereck commented 1 year ago

Also I am hopefully able to run Xorg on i915 again

You might need a custom xorg.conf. x11docker has an undocumented and experimental option --xorgconf. You could work with something like --xorg --xorgconf=FILE.

awerlang commented 1 year ago

I need to reassess what's working and what's not. So far I got some things working trying to get container to use Intel GPU:

Couldn't make it work yet:

mviereck commented 1 year ago

Run apps with --gpu (Xorg on host loads modesetting driver on Intel, while Nvidia is not loaded by xorg) This is what I'd expect from my --gpu=xorg suggestion

Do you use option --xorgconf?

GLX works (glxinfo, glxgears appear to use Intel/Mesa)

I provide an image x11docker/check on docker hub that offers a section to check the GPU. https://hub.docker.com/r/x11docker/check

Run sway with --gpu, even masking /proc/modules.

I have no idea. Any error messages?

Hardware acceleration for video decoding (maybe it's missing a package inside container?)

Video decoding needs additional drivers. Have a look at https://github.com/mviereck/x11docker/wiki/dependencies#dependencies-in-image They might need to be the same version as on host.

--xorg fails opening a tty, likely due to my system's configuration

Run x11docker as root or check the setup described in https://github.com/mviereck/x11docker/wiki/Setup-for-option---xorg

awerlang commented 1 year ago

No, I don't use a --xorgconf (I tried to but didn't fix the outstanding issues). Also, nothing stands out from running x11docker/check. I checked the wiki for missing packages but it's all there for an Ubuntu image. They are probably not the same versions as the host but close enough.

For sway this is what I get:

$ x11docker --backend=podman --xc --init=systemd --home --share /etc/machine-id --weston-xwayland --share=/dev/dri --group-add=video --group-add=render -- --privileged --security-opt mask=/proc/modules -- ${IMAGE}

00:00:00.052 [ERROR] [wlr] [types/wlr_drm_lease_v1.c:715] No DRM backend supplied, failed to create wlr_drm_lease_v1_manager 00:00:00.124 [ERROR] [wlr] [backend/x11/backend.c:695] X11 error: op ChangeProperty (no minor), code Atom (no extension), sequence 63, value 0 sway: backend/x11/output.c:682: handle_x11_present_event: Assertion `buffer->n_busy > 0' failed. 2023-02-20 21:54:01 - [main.c:582] wl_display_roundtrip failed /x11docker/cmdrc: line 92: 1058 Aborted (core dumped) sway