OGRE2: Gazebo (Fortress) rendering with OGRE2 crashes in a WSL2 environment with a dedicated GPU present, aka WSLg

DarthBubi commented 2 years ago

Gazebo crashes when initialising the rendering while using OGRE2 as a rendering output in WSL2. Changing to OGRE1 works, since Gazebo Classic also uses this engine and works flawlessly. I checked against outdated binary builds by compiling the current code, but the behaviour is the same. It might be a problem in D3D12 kernel driver or in the way OGRE2 uses OpenGL, but since I'm no expert, I didn't dig too deep. The same behaviour might also apply to a WSL2 environment on Windows 10, but I have to test that.

Environment

OS Version: WSL Ubuntu 20.04 on Windows 11 (21H2)
binary (fortress) and source (ign-rendering6 and ign-gazebo6)
If this is a GUI or sensor rendering bug, describe your GPU and rendering system. Otherwise delete this section.
- Rendering plugin: ogre2.
  - [ ] Sensor rendering error.
  - [ ] GUI rendering error.
- EGL headless mode:
  - [ ] Running in EGL headless mode
- Generally, mention all circumstances that might affect rendering capabilities:
  - [x] running on a dual GPU machine (integrated GPU + discrete GPU)
  - [ ] running on a multi-GPU machine (it has multiple discrete GPUs)
  - [ ] running on real hardware
  - [x] running in virtual machine
  - [ ] running in Docker/Singularity
  - [ ] running remotely (e.g. via SSH)
  - [ ] running in a cloud
  - [ ] using VirtualGL, XVFB, Xdummy, XVNC or other indirect rendering utilities
  - [x] GPU is concurrently used for other tasks
  - [ ] desktop acceleration
  - [x] video decoding (i.e. a playing Youtube video)
  - [ ] video encoding
  - [ ] CUDA/ROCm computations (Tensorflow, Torch, Caffe running)
  - [ ] multiple simulators running at the same time
  - [ ] other...
- Rendering system info:
  - On Linux, provide the outputs of the following commands:
```
LANG=C glxinfo -B | grep -i '\(direct rendering\|opengl\|profile\)'  # might require installing mesa-utils package
direct rendering: Yes
  Preferred profile: core (0x1)
  Max core profile version: 4.2
  Max compat profile version: 4.2
  Max GLES1 profile version: 1.1
  Max GLES[23] profile version: 3.1
OpenGL vendor string: Microsoft Corporation
OpenGL renderer string: D3D12 (NVIDIA GeForce MX150)
OpenGL core profile version string: 4.2 (Core Profile) Mesa 22.1.2 - kisak-mesa PPA
OpenGL core profile shading language version string: 4.20
OpenGL core profile context flags: (none)
OpenGL core profile profile mask: core profile
OpenGL version string: 4.2 (Compatibility Profile) Mesa 22.1.2 - kisak-mesa PPA
OpenGL shading language version string: 4.20
OpenGL context flags: (none)
OpenGL profile mask: compatibility profile
OpenGL ES profile version string: OpenGL ES 3.1 Mesa 22.1.2 - kisak-mesa PPA
OpenGL ES profile shading language version string: OpenGL ES GLSL ES 3.10
```
- ogre2.log

``` terminate called after throwing an instance of 'Ogre::UnimplementedException' what(): OGRE EXCEPTION(9:UnimplementedException): in GL3PlusTextureGpu::copyTo at /var/lib/jenkins/workspace/ogre-2.2-debbuilder/repo/RenderSystems/GL3Plus/src/OgreGL3PlusTextureGpu.cpp (line 677) Stack trace (most recent call last) in thread 17474: #9 Object "[0xffffffffffffffff]", at 0xffffffffffffffff, in #8 Object "/lib/x86_64-linux-gnu/libc.so.6", at 0x7f0beff6d132, in clone #7 Object "/lib/x86_64-linux-gnu/libpthread.so.0", at 0x7f0befe33608, in #6 Object "/lib/x86_64-linux-gnu/libQt5Core.so.5", at 0x7f0bea1eb96d, in #5 Object "/lib/x86_64-linux-gnu/libQt5Core.so.5", at 0x7f0bea1e9e8a, in qTerminate() #4 Object "/lib/x86_64-linux-gnu/libstdc++.so.6", at 0x7f0bec1e43f6, in std::terminate() #3 Object "/lib/x86_64-linux-gnu/libstdc++.so.6", at 0x7f0bec1e438b, in #2 Object "/lib/x86_64-linux-gnu/libstdc++.so.6", at 0x7f0bec1d8910, in #1 Object "/lib/x86_64-linux-gnu/libc.so.6", at 0x7f0befe70858, in abort #0 Object "/lib/x86_64-linux-gnu/libc.so.6", at 0x7f0befe9100b, in gsignal Aborted (Signal sent by tkill() 17412 1000) ```

Description

Expected behavior: Rendering of the shapes world
Actual behavior: Crash while doing so

Steps to reproduce

Start Gazebo with ign gz --render-engine ogre2 shapes.sdf -v4 in a WSL2(g) environment with a dedicated GPU present

Output

auto_default.log

iche033 commented 2 years ago

rosiakpiotr commented 1 year ago

Any possible updates on this one?

iche033 commented 1 year ago

I think the workaround is to do software rendering by setting the LIBGL_ALWAYS_SOFTWARE=1 env var as suggested in https://github.com/gazebosim/gz-sim/issues/920#issuecomment-1279725124

hamishwillee commented 1 year ago

This also occurs in WSL2 running Ubuntu 22.02 with Gazebo Garden. The error says:

libEGL warning: failed to open /dev/dri/renderD128: Permission denied

LIBGL_ALWAYS_SOFTWARE=1 "works" but is unusably slow. At least for PX4 development you can also do HEADLESS=1, but that gives you the simulation without the UI.

This will make PX4 development using Garden on Windows impossible.

traversaro commented 1 year ago

@iche033 do you think we can either close this one or https://github.com/gazebosim/gz-sim/issues/920#issuecomment-1279725124, to keep just one issue to track Gazebo Sim support on WSL with GPU-accelerated graphics?

@rosiakpiotr @hamishwillee I guess you can check https://github.com/gazebosim/gz-sim/issues/920 for the latest updates on this issue. On WSL2 with GPU support, if anyone is interested in contributing the issue to solve a possible path forward is to use GL_NV_copy_image instead of GL_ARB_copy_image , but are some shader compilation problems described in https://github.com/gazebosim/gz-sim/issues/920#issuecomment-1474487588, that unfortunatly I did not had the time to debug.

On a related note, if you are interested on Gazebo Sim support on Windows native (i.e. without WSL), you can also check https://github.com/gazebosim/gz-sim/issues/168 .

hamishwillee commented 1 year ago

Thanks @traversaro ! FWIW Not all that interested in Gazebo Sim support on windows native right now, because the rest of the PX4 Windows toolchain is on WSL2. @bperseghetti suggested Vulkan might work, but there are no obvious setup instructions for WSL and the issues that show up on google search appear to be still open with lots of problem edge cases.

iche033 commented 1 year ago

@iche033 do you think we can either close this one or https://github.com/gazebosim/gz-sim/issues/920#issuecomment-1279725124, to keep just one issue to track Gazebo Sim support on WSL with GPU-accelerated graphics?

sounds good. Closing this issue to keep all WSL + GPU support conversations in once place: https://github.com/gazebosim/gz-sim/issues/920.

gazebosim / gz-rendering