search

microsoft / WSL

Issues found on WSL
https://docs.microsoft.com/windows/wsl
MIT License
17.23k stars 808 forks source link

WSL2 + Docker + OpenGL + NVIDIA not working (uses llvmpipe) #7507

Open riv-robot opened 2 years ago

riv-robot commented 2 years ago

Windows Build Number

Microsoft Windows [Version 10.0.22000.194]

WSL Version

Kernel Version

5.10.60.1

Distro Version

Ubuntu 20.04

Other Software

Docker Desktop (Windows): 4.1.0 (69386)

Repro Steps

I am running ROS GUI applications like RViz and Gazebo through a docker container on WSL2. The OpenGL renderer is not selecting my NVIDIA GTX 1050 card and uses llvmpipe (CPU) instead.

The OpenGL renderer does find my NVIDIA card outside of a docker container on WSL2 (on the host).

I have replicated the same issue after multiple reinstalls and using docker-ce instead of docker desktop.

On a native Ubuntu 20.04 boot, the containers OpenGL renderer is correctly set to my NVIDIA card.

  1. The image is a custom one, based on the ros-noetic image.
  2. The NVIDIA card is used for OpenGL rendering in the docker container on a native Ubuntu 20.04 install
  3. However it doesn't work with the same image on WSL2
  4. I have tried various NVIDIA images and running glxgears, glxinfo and glmark2
  5. NVIDIA_DRIVER_CAPABILITIES is set to all
  6. I've also tried using the --gpus 'all,"capabilities=graphics"' parameters

Expected Behavior

RViz, Gazebo, glxgears, glmark2 should all render with 3D hardware acceleration on the NVIDIA GPU.

Actual Behavior

Diagnostic Logs

No response

onomatopellan commented 2 years ago

The team just posted some samples of containers with accelerated OpenGL.

https://github.com/microsoft/wslg/tree/main/samples/container

I think the most important was the env variable LD_LIBRARY_PATH=/usr/lib/wsl/lib

riv-robot commented 2 years ago

@spronovo I've been testing accelerated OpenGL through containers in WSL2. I used your dockerfile's from commit ac6221b. These worked which is awesome! I also managed to get RViz and ROS (robotic operating system) working correctly minus one issue:

Any ideas why this may happen?

riv-robot commented 2 years ago

Anyone tested RViz and meshes using accelerated OpenGL in WSL?

riv-robot commented 2 years ago

This NVIDIA thread details others having the same issue whether inside a container or not

riv-robot commented 2 years ago

Friendly ping to anyone who's had this problem and solved it?

tdnguyen6 commented 2 years ago

Similar problem here. Got WebGL to render with WSLg inside Firefox (not vcxsrv or x410). However, it seemed to use CPU rendering even when my GPU was detected as CPU load was high while FPS was low and laggy (tested with: https://webglsamples.org/aquarium/aquarium.html). I use multiple WSL2 distros on windows 11 (22000.348).

Renderer string from Debian and Ubuntu:

❯  glxinfo | grep 'OpenGL renderer string'
OpenGL renderer string: D3D12 (NVIDIA GeForce GTX 1070)

Renderer string from OpenSuse:

~> glxinfo | grep 'OpenGL renderer string'
OpenGL renderer string: Gallium 0.4 on llvmpipe (LLVM 3.8, 128 bits)

Renderer string from Arch:

❯ glxinfo | grep 'OpenGL renderer string'
OpenGL renderer string: llvmpipe (LLVM 13.0.0, 256 bits)

I have just tested again. Hardware accelerated worked for me but only if:

ClaudioCimarelli commented 1 year ago

The team just posted some samples of containers with accelerated OpenGL.

https://github.com/microsoft/wslg/tree/main/samples/container

I think the most important was the env variable LD_LIBRARY_PATH=/usr/lib/wsl/lib

This is working the solution (for Ubuntu 22 the updated unofficial mesa-drivers are required). Also chmod 666 /dev/dri/* and chmod 666 /dev/dxg are useful to avoid segmentation faults as in NVIDIA/nvidia-docker#715. Glxgears is considerably faster without the acceleration enabled (tenfold more FPS). Any ideas why?

mqt0029 commented 1 year ago

Referencing the OpenGL stuff from this NVIDIA OpenGL image and the Containers.md mentioned by other people here, I pieced this together. This is currently working on Windows 11 and Ubuntu 22.04 on WSL2.

The last two lines on the Dockerfile was from WSLg GPU Selection Wiki.

Dockerfile

FROM ubuntu:jammy

ARG DEBIAN_FRONTEND=noninteractive

SHELL ["/bin/bash", "-c"]

RUN apt-get update && apt-get -y upgrade && apt-get -y install \
    libxext-dev \
    libx11-dev \
    libglvnd-dev \
    libglx-dev \
    libgl1-mesa-dev \
    libgl1-mesa-glx \
    libgl1-mesa-dri \
    libegl1-mesa-dev \
    libgles2-mesa-dev \
    freeglut3-dev \
    mesa-utils \
    mesa-utils-extra \
    && apt-get -y autoremove \
    && apt-get clean

ENV LD_LIBRARY_PATH=/usr/lib/wsl/lib
ENV LIBVA_DRIVER_NAME=d3d12

Test Script

#!/usr/bin/env bash
export MESA_D3D12_DEFAULT_ADAPTER_NAME=NVIDIA

docker run \
--device /dev/dxg \
--device /dev/dri/card0 \
--device /dev/dri/renderD128 \
--env DISPLAY=$DISPLAY \
--env WAYLAND_DISPLAY=$WAYLAND_DISPLAY \
--env XDG_RUNTIME_DIR=$XDG_RUNTIME_DIR \
--env PULSE_SERVER=$PULSE_SERVER \
--volume /tmp/.X11-unix:/tmp/.X11-unix \
--volume /mnt/wslg:/mnt/wslg \
--volume /usr/lib/wsl:/usr/lib/wsl \
wsl2-opengl:latest \
glxinfo | grep "OpenGL"

Output

OpenGL vendor string: Microsoft Corporation
OpenGL renderer string: D3D12 (NVIDIA GeForce RTX 2080)
OpenGL core profile version string: 4.2 (Core Profile) Mesa 23.0.4-0ubuntu1~22.04.1
OpenGL core profile shading language version string: 4.20
OpenGL core profile context flags: (none)
OpenGL core profile profile mask: core profile
OpenGL core profile extensions:
OpenGL version string: 4.2 (Compatibility Profile) Mesa 23.0.4-0ubuntu1~22.04.1
OpenGL shading language version string: 4.20
OpenGL context flags: (none)
OpenGL profile mask: compatibility profile
OpenGL extensions:
OpenGL ES profile version string: OpenGL ES 3.1 Mesa 23.0.4-0ubuntu1~22.04.1
OpenGL ES profile shading language version string: OpenGL ES GLSL ES 3.10
OpenGL ES profile extensions:
elgarbe commented 12 months ago

Hi, I've tried your dockerfile in wsl2:

docker build --rm -t "wsl2-opengl:latest" . and docker run --device /dev/dxg --gpus all --env DISPLAY=$DISPLAY --env WAYLAND_DISPLAY=$WAYLAND_DISPLAY --env XDG_RUNTIME_DIR=$XDG_RUNTIME_DIR --env PULSE_SERVER=$PULSE_SERVER --volume /tmp/.X11-unix:/tmp/.X11-unix --volume /mnt/wslg:/mnt/wslg --volume /usr/lib/wsl:/usr/lib/wsl wsl2-opengl:latest glxinfo | grep "OpenGL"

but I get an error:

Error: unable to open display

mqt0029 commented 11 months ago

Hi, I've tried your dockerfile in wsl2:

docker build --rm -t "wsl2-opengl:latest" . and docker run --device /dev/dxg --gpus all --env DISPLAY=$DISPLAY --env WAYLAND_DISPLAY=$WAYLAND_DISPLAY --env XDG_RUNTIME_DIR=$XDG_RUNTIME_DIR --env PULSE_SERVER=$PULSE_SERVER --volume /tmp/.X11-unix:/tmp/.X11-unix --volume /mnt/wslg:/mnt/wslg --volume /usr/lib/wsl:/usr/lib/wsl wsl2-opengl:latest glxinfo | grep "OpenGL"

but I get an error:

Error: unable to open display

As it turns out I completely misunderstood the GPU selection. I have revised the answer. On the off chance that it still not work, there is a couple of things you can do:

1. Verify your WSL2 setup can actually spawn GUI apps

user@host:~$ sudo apt -y install mesa-utils
user@host:~$ glxgears

You should see a window pops up with a set of colored gear spinning.

2. Verify that your NVIDIA GPU is being utilized for the correct DISPLAY variable

user@host:~$ export MESA_D3D12_DEFAULT_ADAPTER_NAME=NVIDIA
user@host:~$ glxinfo -B
name of display: :0
display: :0  screen: 0
direct rendering: Yes
Extended renderer info (GLX_MESA_query_renderer):
    Vendor: Microsoft Corporation (0xffffffff)
    Device: D3D12 (NVIDIA GeForce RTX 3050 Ti Laptop GPU) (0xffffffff)
    Version: 22.0.5
    Accelerated: yes
    Video memory: 20279MB
    Unified memory: no
    Preferred profile: core (0x1)
    Max core profile version: 3.3
    Max compat profile version: 3.3
    Max GLES1 profile version: 1.1
    Max GLES[23] profile version: 3.1
OpenGL vendor string: Microsoft Corporation
OpenGL renderer string: D3D12 (NVIDIA GeForce RTX 3050 Ti Laptop GPU)
OpenGL core profile version string: 3.3 (Core Profile) Mesa 22.0.5
OpenGL core profile shading language version string: 3.30
OpenGL core profile context flags: (none)
OpenGL core profile profile mask: core profile

OpenGL version string: 3.3 (Compatibility Profile) Mesa 22.0.5
OpenGL shading language version string: 3.30
OpenGL context flags: (none)
OpenGL profile mask: compatibility profile

OpenGL ES profile version string: OpenGL ES 3.1 Mesa 22.0.5
OpenGL ES profile shading language version string: OpenGL ES GLSL ES 3.10

3. Try updating your kernel through Powershell

PS> wsl --shutdown
PS> wsl --update

And try spawning the GUI window again.

elgarbe commented 11 months ago

runing glxgears in wsl result on a windows pop-up with the gears, but the Intel GPU is used instead of Nvidia one. Is that right? EDIT: I'm sorry the "export MESA_D3D12_DEFAULT_ADAPTER_NAME=NVIDIA" do the job. Running that line before glxgears makes NVIDIA GPU used.

mqt0029 commented 11 months ago

runing glxgears in wsl result on a windows pop-up with the gears, but the Intel GPU is used instead of Nvidia one. Is that right? EDIT: I'm sorry the "export MESA_D3D12_DEFAULT_ADAPTER_NAME=NVIDIA" do the job. Running that line before glxgears makes NVIDIA GPU used.

Great! It means that the GPU is visible and should be able to be passed into the container, check internally from the container terminal that glxinfo | grep "OpenGL" outputs something along the line of

user@host:~$ glxinfo | grep "OpenGL"
OpenGL vendor string: NVIDIA Corporation
OpenGL renderer string: Quadro P4000/PCIe/SSE2
OpenGL core profile version string: 4.6.0 NVIDIA 535.113.01
OpenGL core profile shading language version string: 4.60 NVIDIA
OpenGL core profile context flags: (none)
OpenGL core profile profile mask: core profile
OpenGL core profile extensions:
OpenGL version string: 4.6.0 NVIDIA 535.113.01
OpenGL shading language version string: 4.60 NVIDIA
OpenGL context flags: (none)
OpenGL profile mask: (none)
OpenGL extensions:
OpenGL ES profile version string: OpenGL ES 3.2 NVIDIA 535.113.01
OpenGL ES profile shading language version string: OpenGL ES GLSL ES 3.20
OpenGL ES profile extensions: