search

neutrinolabs / xrdp

xrdp: an open source RDP server
http://www.xrdp.org/
Apache License 2.0
5.74k stars 1.73k forks source link

Get XRDP with Nvidia Acceleration working on Microsoft WSL2 #2328

Open Nexarian opened 2 years ago

Nexarian commented 2 years ago

I've been playing with getting this to work while afflicted by Covid. The good news is that it does appear that nvidia-smi on Ubuntu 20.04 with WSL2 on Windows 10 works, but I still can't get XRDP itself to load, currently blocked by this error, which is like a beast that can never be killed, only suspended for a bit:

[    17.434] rdpkeybPlug:
[    17.434] (II) modesetting: Driver for Modesetting Kernel Drivers: kms
[    17.434] (EE)
Fatal server error:
[    17.434] (EE) parse_vt_settings: Cannot open /dev/tty0 (Permission denied)
[    17.434] (EE)
[    17.434] (EE)
Please consult the The X.Org Foundation support
         at http://wiki.x.org
 for help.
[    17.434] (EE) Please also check the log file at ".xorgxrdp.10.log" for additional information.
[    17.434] (EE)
[    17.434] (WW) xf86CloseConsole: KDSETMODE failed: Bad file descriptor
[    17.434] (WW) xf86CloseConsole: VT_GETMODE failed: Bad file descriptor
[    17.434] (EE) Server terminated with error (1). Closing log file.

The other note, you can't use lspci to get the BUS_ID of the Nvidia video card because of the underlying HyperV system, so you have to ask the Nvidia driver directly what its bus ID is:

nvidia-smi --query-gpu=gpu_bus_id --format=csv,noheader | cut -d":" -f2

Those who attempt this, please put your notes and findings here! Making this work will speed up XRDP development, as it means that anyone who has a Windows host can also develop XRDP SxS with the same hardware.

Nexarian commented 2 years ago

@jsorg71 FYI

Nexarian commented 2 years ago

Adding the following to /etc/X11/xrdp/xorg_nvidia.conf helps load the Nvidia driver:

Section "Files"
    ModulePath "/usr/lib/x86_64-linux-gnu/nvidia/xorg/"
    ModulePath "/usr/lib/xorg/modules"
EndSection

But still stuck on a crash for loading the modesetting driver.

Nexarian commented 2 years ago 
[    48.327]
X.Org X Server 1.20.13
X Protocol Version 11, Revision 0
[    48.327] Build Operating System: linux Ubuntu
[    48.327] Current Operating System: Linux RAINBOW 5.10.102.1-microsoft-standard-WSL2 #1 SMP Wed Mar 2 00:30:59 UTC 2022 x86_64
[    48.327] Kernel command line: initrd=\initrd.img panic=-1 pty.legacy_count=0 nr_cpus=32 nomodeset nouveau.modeset=0
[    48.327] Build Date: 06 July 2022  01:53:24PM
[    48.327] xorg-server 2:1.20.13-1ubuntu1~20.04.3 (For technical support please see http://www.ubuntu.com/support)
[    48.327] Current version of pixman: 0.38.4
[    48.327]    Before reporting problems, check http://wiki.x.org
        to make sure that you have the latest version.
[    48.327] Markers: (--) probed, (**) from config file, (==) default setting,
        (++) from command line, (!!) notice, (II) informational,
        (WW) warning, (EE) error, (NI) not implemented, (??) unknown.
[    48.327] (++) Log file: ".xorgxrdp.10.log", Time: Sun Aug  7 01:22:18 2022
[    48.328] (++) Using config file: "/etc/X11/xrdp/xorg_nvidia.conf"
[    48.328] (==) Using system config directory "/usr/share/X11/xorg.conf.d"
[    48.328] (**) Option "defaultserverlayout" "XRDP GPU Server"
[    48.328] (**) ServerLayout "XRDP GPU Server"
[    48.328] (**) |-->Screen "dGPU" (0)
[    48.328] (**) |   |-->Monitor "<default monitor>"
[    48.329] (**) |   |-->Device "dGPU"
[    48.329] (==) No monitor specified for screen "dGPU".
        Using a default monitor configuration.
[    48.329] (**) |-->Input Device "xrdpMouse"
[    48.329] (**) |-->Input Device "xrdpKeyboard"
[    48.329] (**) Option "DontVTSwitch" "on"
[    48.329] (**) Option "AutoAddDevices" "off"
[    48.329] (**) Not automatically adding devices
[    48.329] (==) Automatically enabling devices
[    48.329] (==) Automatically adding GPU devices
[    48.329] (==) Automatically binding GPU devices
[    48.329] (==) Max clients allowed: 256, resource mask: 0x1fffff
[    48.330] (WW) The directory "/usr/share/fonts/X11/cyrillic" does not exist.
[    48.330]    Entry deleted from font path.
[    48.330] (WW) The directory "/usr/share/fonts/X11/100dpi/" does not exist.
[    48.330]    Entry deleted from font path.
[    48.330] (WW) The directory "/usr/share/fonts/X11/75dpi/" does not exist.
[    48.330]    Entry deleted from font path.
[    48.331] (WW) The directory "/usr/share/fonts/X11/100dpi" does not exist.
[    48.331]    Entry deleted from font path.
[    48.331] (WW) The directory "/usr/share/fonts/X11/75dpi" does not exist.
[    48.331]    Entry deleted from font path.
[    48.331] (==) FontPath set to:
        /usr/share/fonts/X11/misc,
        /usr/share/fonts/X11/Type1,
        built-ins
[    48.331] (**) ModulePath set to "/usr/lib/x86_64-linux-gnu/nvidia/xorg/,/usr/lib/xorg/modules"
[    48.331] (II) Loader magic: 0x562cd0ceb020
[    48.331] (II) Module ABI versions:
[    48.331]    X.Org ANSI C Emulation: 0.4
[    48.331]    X.Org Video Driver: 24.1
[    48.331]    X.Org XInput driver : 24.1
[    48.331]    X.Org Server Extension : 10.0
[    48.331] (EE) dbus-core: error connecting to system bus: org.freedesktop.DBus.Error.FileNotFound (Failed to connect to socket /var/run/dbus/system_bus_socket: No such file or directory)
[    48.332] (--) PCI:*(0@5484:0:0) 1414:008e:0000:0000 rev 0
[    48.332] (II) "glx" will be loaded by default.
[    48.332] (II) LoadModule: "xorgxrdp"
[    48.334] (II) Loading /usr/lib/xorg/modules/libxorgxrdp.so
[    48.335] (II) Module XORGXRDP: vendor="X.Org Foundation"
[    48.335]    compiled for 1.20.13, module version = 0.9.80
[    48.335]    ABI class: X.Org Video Driver, version 24.1
[    48.335] xorgxrdpSetup:
[    48.335] (II) LoadModule: "glx"
[    48.335] (II) Loading /usr/lib/xorg/modules/extensions/libglx.so
[    48.341] (II) Module glx: vendor="X.Org Foundation"
[    48.341]    compiled for 1.20.13, module version = 1.0.0
[    48.341]    ABI class: X.Org Server Extension, version 10.0
[    48.341] (II) LoadModule: "nvidia"
[    48.341] (II) Loading /usr/lib/x86_64-linux-gnu/nvidia/xorg/nvidia_drv.so
[    48.343] (II) Module nvidia: vendor="NVIDIA Corporation"
[    48.343]    compiled for 1.6.99.901, module version = 1.0.0
[    48.343]    Module class: X.Org Video Driver
[    48.343] (II) LoadModule: "xrdpmouse"
[    48.343] (II) Loading /usr/lib/xorg/modules/input/xrdpmouse_drv.so
[    48.343] (II) Module XRDPMOUSE: vendor="X.Org Foundation"
[    48.343]    compiled for 1.20.13, module version = 0.9.80
[    48.343]    Module class: X.Org XInput Driver
[    48.343]    ABI class: X.Org XInput driver, version 24.1
[    48.343] rdpmousePlug:
[    48.343] (II) LoadModule: "xrdpkeyb"
[    48.344] (II) Loading /usr/lib/xorg/modules/input/xrdpkeyb_drv.so
[    48.344] (II) Module XRDPKEYB: vendor="X.Org Foundation"
[    48.344]    compiled for 1.20.13, module version = 0.9.80
[    48.344]    Module class: X.Org XInput Driver
[    48.344]    ABI class: X.Org XInput driver, version 24.1
[    48.344] rdpkeybPlug:
[    48.344] xorgxrdpCheckWrap: NVIDIA driver found
[    48.344] (II) NVIDIA dlloader X Driver  515.65.01  Wed Jul 20 13:50:46 UTC 2022
[    48.344] (II) NVIDIA Unified Driver for all Supported NVIDIA GPUs
[    48.344] xorgxrdpDriverFunc:
[    48.344] (EE) No devices detected.
[    48.344] (==) Matched modesetting as autoconfigured driver 0
[    48.344] (==) Matched fbdev as autoconfigured driver 1
[    48.344] (==) Matched vesa as autoconfigured driver 2
[    48.344] (==) Assigned the driver to the xf86ConfigLayout
[    48.344] (II) LoadModule: "modesetting"
[    48.344] (II) Loading /usr/lib/xorg/modules/drivers/modesetting_drv.so
[    48.345] (II) Module modesetting: vendor="X.Org Foundation"
[    48.345]    compiled for 1.20.13, module version = 1.20.13
[    48.345]    Module class: X.Org Video Driver
[    48.345]    ABI class: X.Org Video Driver, version 24.1
[    48.345] (II) LoadModule: "fbdev"
[    48.345] (II) Loading /usr/lib/xorg/modules/drivers/fbdev_drv.so
[    48.345] (II) Module fbdev: vendor="X.Org Foundation"
[    48.345]    compiled for 1.20.1, module version = 0.5.0
[    48.345]    Module class: X.Org Video Driver
[    48.345]    ABI class: X.Org Video Driver, version 24.0
[    48.345] (II) LoadModule: "vesa"
[    48.345] (WW) Warning, couldn't open module vesa
[    48.345] (EE) Failed to load module "vesa" (module does not exist, 0)
[    48.345] (II) LoadModule: "xrdpmouse"
[    48.346] (II) Loading /usr/lib/xorg/modules/input/xrdpmouse_drv.so
[    48.346] (II) Module XRDPMOUSE: vendor="X.Org Foundation"
[    48.346]    compiled for 1.20.13, module version = 0.9.80
[    48.346]    Module class: X.Org XInput Driver
[    48.346]    ABI class: X.Org XInput driver, version 24.1
[    48.346] rdpmousePlug:
[    48.346] (II) LoadModule: "xrdpkeyb"
[    48.346] (II) Loading /usr/lib/xorg/modules/input/xrdpkeyb_drv.so
[    48.346] (II) Module XRDPKEYB: vendor="X.Org Foundation"
[    48.346]    compiled for 1.20.13, module version = 0.9.80
[    48.346]    Module class: X.Org XInput Driver
[    48.346]    ABI class: X.Org XInput driver, version 24.1
[    48.346] rdpkeybPlug:
[    48.346] (II) NVIDIA dlloader X Driver  515.65.01  Wed Jul 20 13:50:46 UTC 2022
[    48.346] (II) NVIDIA Unified Driver for all Supported NVIDIA GPUs
[    48.346] xorgxrdpDriverFunc:
[    48.346] (II) modesetting: Driver for Modesetting Kernel Drivers: kms
[    48.346] (II) FBDEV: driver for framebuffer: fbdev
[    48.346] (EE)
Fatal server error:
[    48.346] (EE) parse_vt_settings: Cannot open /dev/tty0 (Permission denied)
[    48.346] (EE)
[    48.346] (EE)
Please consult the The X.Org Foundation support
         at http://wiki.x.org
 for help.
[    48.346] (EE) Please also check the log file at ".xorgxrdp.10.log" for additional information.
[    48.346] (EE)
[    48.346] (WW) xf86CloseConsole: KDSETMODE failed: Bad file descriptor
[    48.346] (WW) xf86CloseConsole: VT_GETMODE failed: Bad file descriptor
[    48.346] (EE) Server terminated with error (1). Closing log file.
Nexarian commented 2 years ago

It's as if, even though the nvidia driver is detected properly, something in Xorg decides not to proceed with loading the driver. I tried disabling modesetting by editing %HOMEPATH%/.wslconfig and adding:

# Settings apply across all Linux distros running on WSL 2
[wsl2]

# Sets additional kernel parameters, in this case enabling older Linux base images such as Centos 6
kernelCommandLine = nomodeset nouveau.modeset=0

But it appeared to do nothing in spite of the fact that cat /proc/cmdline shows:

initrd=\initrd.img panic=-1 pty.legacy_count=0 nr_cpus=32 nomodeset nouveau.modeset=0
Nexarian commented 2 years ago

Another note: XRDP does work if I switch it to non-Nvidia-accelerated. That is using xorg.conf instead of xorg_nvidia.conf, so this is not a problem with any other part of XRDP infra. It's specifically Nvidia driver related.

ismaell commented 1 year ago

Does the Linux kernel inside WSL2 get access to a NVIDIA vGPU from the Windows kernel? otherwise this won't work...