search

VirtualGL / virtualgl

Main VirtualGL repository
https://VirtualGL.org
Other
701 stars 106 forks source link

Docker + xrdp not working using NVidia #180

Closed KenYN closed 3 years ago

KenYN commented 3 years ago

I'm using version 2.6.5 built under Docker with Ubuntu 20.04 using the option cmake -DVGL_BUILDSERVER=ON .. and running from a remote Windows 10 client running over xrdp using the standard Windows Remote Desktop app and am trying to accelerate an OpenGL-based app.

On Friday when I accessed Docker directly from the Ubuntu device the program ran happily with DISPLAY=:10 /opt/VirtualGL/bin/vglrun rviz2 and the accelerated display (looking at htop processor usage) appeared on my Windows device as expected.

Today I am WFH, and I need the extra -d :10 command line option - DISPLAY=:10 /opt/VirtualGL/bin/vglrun -d :10 rviz2 - to get the code to run, but it is not accelerated. I've tried launching from a terminal in VSCode, a PuTTY command line, and a terminal in the xrdp session, but all without success. What am I doing wrong? I've read various other reports here for getting NVidia working, but I've not managed to get it to work remotely.

EDIT: Just tried again installing virtualgl_2.6.5_amd64.deb and acceleration works! What are the build flags needed to reproduce the deb?

dcommander commented 3 years ago

DISPLAY and VGL_DISPLAY/vglrun -d should never be set to the same value. VGL_DISPLAY/vglrun -d should point to the "3D X server", an X server that is attached to a GPU. DISPLAY should point to the "2D X server", which is normally either an X proxy instance (Xvnc, Xpra, FreeNX, etc.) running on the VirtualGL server or an X server running on the client. (Refer to the VirtualGL User's Guide for diagrams.)

I have virtually no experience with Docker on Windows, but I can tell you that your basic strategy will be to either:

  1. figure out how to run a GPU-attached X server on the host and access that X server from within a Docker container or
  2. figure out how to access the GPU device from within a Docker container so you can run a GPU-attached X server in the container.

On Linux, Option 1 generally involves forwarding the Unix domain socket for the 3D X server from host to container and using the nVidia Container Toolkit to forward DRI devices from host to container. (See examples here.) However, I'm not sure if that can be done on Windows, so Option 2 may be your only option.

Presumably you have already figured out how to access the GPU within the Docker container, since you were able to get VirtualGL to work with GPU acceleration while you were sitting in front of the host machine. (NOTE: Setting VGL_LOGO=1 is the easiest way to confirm that VirtualGL is loaded and working with a particular application, and you can also examine the OpenGL renderer string in the output of vglrun glxinfo to verify that OpenGL is using a GPU renderer rather than a software renderer such as llvmpipe.) The issue with Remote Desktop may be due to the fact that Remote Desktop doesn't support accelerated OpenGL-- at least not by default. I read that you can enable accelerated OpenGL within Remote Desktop on Windows 10 by editing a group policy, but I haven't tried that, so I can't confirm that it works. (I don't actually have Windows 10 running in my lab except in a virtual machine environment, because I only ever need to test Windows as a client. Thus, I have no way of duplicating your configuration.) From the point of view of VirtualGL, the container is the "VirtualGL server" and the host is the "client", so if you can figure out how to access the GPU from within the Remote Desktop session and then within the container, the rest of the VirtualGL User's Guide should apply. However, Windows servers are outside the scope of free/community support, and I am not aware that anyone has ever successfully accomplished what you are attempting to accomplish.

As far as reproducing the .deb, please see https://github.com/VirtualGL/buildscripts for the official build scripts and https://github.com/VirtualGL/docker for the recipe file used to generate our official Docker build container.