495 symlinks

[FakeShemp]

The new 495 beta driver seems to need some files symlinked to work properly.

Details are here: https://www.nvidia.com/download/driverResults.aspx/181167/en

Currently this happens:

MESA-LOADER: failed to open nvidia-drm: /usr/lib/x86_64-linux-gnu/GL/default/lib/gbm/nvidia-drm_gbm.so: cannot open shared object file: No such file or directory (search paths /usr/lib/x86_64-linux-gnu/GL/default/lib/gbm, suffix _gbm)

[doraskayo]

Can you try using the GBM_BACKENDS_PATH environment variable to point at the current directory where nvidia-drm_gbm.so is located?

[gasinvein]

There is no nvidia-drm_gbm.so yet, it should be created at install time.

[gasinvein]

I've opened #77 that should create the symlink.

[gasinvein]

The update with added symlink has landed. The nvidia-drm_gbm.so file should be in /usr/lib/x86_64-linux-gnu/GL/nvidia-495-29-05/lib/gbm/. @FakeShemp you can now test what @doraskayo suggested.

[FakeShemp]

@gasinvein I don't get the MESA-LOADER error anymore, so the symlink seems to be working fine. The GBM backend seems to need something more to actually initialize though.

[gasinvein]

@FakeShemp Do gpu-accelerated flatpak apps (e.g. games) work at all? AFAIK full support for GBM in Nvidia drivers requires some work in wayland compositors that supported EGLStreams (e.g. Mutter).

[FakeShemp]

Running a GNOME x11 session, everything seems to work just like it should and I can get GPU acceleration.

Under Wayland session, neither xwayland nor pure wayland seem to be able to use GPU acceleration when running a flatpak. Non-flatpaked apps seem to work when not using xwayland, but I will try with 21.1.2.901 and see if that helps.

I'm very uneducated in how these components interact, so I'm having difficulty understanding where the roadblocks are.

[FakeShemp]

@gasinvein All right, with xwayland 21.1.2.901 I can get GPU acceleration with Flatpak apps on x11, during a Wayland session.

[gasinvein]

So, GPU acceleration doesn't work for native wayland apps in flatpak only, correct?

[FakeShemp]

Yes, that is correct.

[gasinvein]

Ok, I guess nothing else needs to be (or can be) done on this GL extension side, so the issue ~should be forwarded to freedesktop-sdk~ ah, it's already there.

[gasinvein]

@FakeShemp The driver update with required libs has landed in flathub repo. Can you please try running apps with this environment variable?

__EGL_EXTERNAL_PLATFORM_CONFIG_DIRS=/usr/lib/x86_64-linux-gnu/GL/nvidia-495-29-05/egl/egl_external_platform.d/

(replace 495-29-05 with 495-44 if you've already updated to it)

[gasinvein]

Also posting you setup that you use to reproduce this issue (e.g. distro, compositor, the gpu-using app) may be helpful.

[FakeShemp]

I'm using fedora rawhide with an rtx 3070 ti.

Gnome 41 with wayland.

Mainly testing with Brave, Lutris and GPU-Viewer.

[gasinvein]

For those who missed the discussion in the matrix room, a brief: While investigating this issue, we've discovered that hardware 3D acceleration doesn't work at all in flatpak apps using native Wayland on Nvidia GPUs (and most likely it never worked), both with EGLStreams and (most likely) with GBM. This appears to be caused by missing egl-wayland, a separate library that wasn't available in flatpak sandbox, while recent Nvidia drivers depend on it. egl-wayland libraries and ICDs were added in #78. Hardware 3D should work now if the ICD paths are set up (currently you have to set them up manually, this needs some tweaks on freedesktop-sdk side to make it work OOTB). However, I didn't manage to run a GBM-only compositor (like Sway or Weston) on the GBM-enabled Nvidia drivers, so tested the hardware acceleration only on GNOME, which supposedly still uses EGLStreams on drivers 495.

[FakeShemp]

@FakeShemp The driver update with required libs has landed in flathub repo. Can you please try running apps with this environment variable?

__EGL_EXTERNAL_PLATFORM_CONFIG_DIRS=/usr/lib/x86_64-linux-gnu/GL/nvidia-495-29-05/egl/egl_external_platform.d/

(replace 495-29-05 with 495-44 if you've already updated to it)

There definitely seem to be something more happening than before. Vulkan rendering seems to work fine. :+1:

GL contexts seem to not do as well yet, but once again I'm not sure why that is exactly.

Trying to run an SDL2 app with OpenGL under Wayland gives the error:

WARNING: SDL_SetVideoMode says we can't switch to that mode (Invalid window)!

[gasinvein]

@FakeShemp Can you please list exact games/apps you're testing what with? For me it was surprisingly hard to find an app that supports native Wayland and makes use of 3D acceleration. I was using GTK4 inspector, since it certainty supports Wayland and can show info about both EGL and Vulkan renderers in use.

[FakeShemp]

I was using Chromium/Brave and checking chrome://gpu with some success. Vulkan acceleration seems to work just fine. OpenGL seems to not want to run at all with Wayland though.

These flags help with enabling and disabling related stuff

--disable-gpu-sandbox
--enable-features=UseOzonePlatform,WebRTCPipeWireCapturer
--enable-features=Vulkan
--ozone-platform=wayland

I was also trying to run some SDL software like ScummVM, but couldn't get it to run under Wayland.

I tried GPU-Viewer to look at its reports too. vulkaninfo will not run with nvidia drivers at all, outside flatpak as well, so I didn't get any Vulkan reports. GL reports seem to rely on x11 but works fine.

Neothesia also seem to have problems running on Wayland with the nvidia drivers.

[FakeShemp]

Looking at RPM Fusion, there also seem to be a libnvidia-vulkan-producer.so lib that might need a symlink to help Vulkan.

https://pkgs.rpmfusion.org/cgit/nonfree/xorg-x11-drv-nvidia.git/commit/?id=8b88d958f28cb28f0fcfef8581adfae565212135

[gasinvein]

As I wrote in the freedesktop-sdk issue comment, I've managed to get the current GL extension to work with Fedora flatpak runtime, with both EGLStreams and GBM compositors. So the issue must be with something in freedesktop-sdk; I'm not sure if the GL extension needs anything else for wayland 3D acceleration on top of what was already added.

[gasinvein]

@FakeShemp I've did some more tests and it appears thet you're right, the libnvidia-vulkan-producer.so symlink is required indeed. Thanks for pointing this out. Although it seems that it's still not enough to get OpenGL working.

[Gert-dev]

So, GPU acceleration doesn't work for native wayland apps in flatpak only, correct?

Is this also the reason that with Wayland on NVIDIA and GBM, the Flatpaked Firefox 96 (using the native Wayland back end, which it uses by default for me) from Flathub falls back to WebRender (Software) and only shows llvmpipe as GPU?

(It works fine when using the non-Flatpak-ed Firefox, as well as on X11 with either the Flatpak-ed or non-Flatpak-ed version.)

EDIT: Strange, this issue has been occurring for me since I started testing with NVIDIA GBM since at least late October, and with Firefox 94 and later versions; I tested it again when Firefox 96 was released and it was still broken. I now updated Flatpak's NVIDIA driver based on this repository to contain the updates from 11 January, and it seems to be fixed now (I'm currently using version 495.46 of the driver, GNOME 41.3, and am of course using the GBM back end).

I don't really understand why, but if it's fixed, it's fixed :+1: . In that case let this serve as additional verification that I get hardware acceleration on GBM Wayland using Flatpak applications - I believe of the OpenGL kind (which I believe is what WebRender uses).

[gasinvein]

It should be fixed in the freedesktop runtime 21.08.9 (released on 10 Jan or so). See this issue.

[prodrigestivill]

I think that freedesktop runtime 21.08.9 and above has it working when using EGL+GBM.

But it fails when using directly libgbm.

As an example it can be done using Steam+gamescope:

flatpak install com.valvesoftware.Steam com.valvesoftware.Steam.Utility.gamescope

Then an example of failing GBM:

flatpak run --command=gamescope com.valvesoftware.Steam -e -- steam -gamepadui

Outputs the follwing error:

...
couldn't create gbm device
EGL setup failed, disabling glamor
Failed to initialize glamor, falling back to sw
...

But setting the GBM_BACKENDS_PATH works:

flatpak run --env="GBM_BACKENDS_PATH=/usr/lib/x86_64-linux-gnu/GL/$(flatpak --gl-drivers | head -1)/extra/gbm/" --command=gamescope com.valvesoftware.Steam -e -- steam -gamepadui

[zdobersek]

With the nvidia-525-89-02 version, the nvidia-drm_gbm.so symlink is set up under /usr/lib/x86_64-linux-gnu/GL/nvidia-525-89-02/extra/gbm/. I too can confirm exporting GBM_BACKENDS_PATH brings things together.

But originally the default gbm backends path is used. That is a configurable option on the libgbm project. This is where the nvidia-drm_gbm.so binary is looked for by default:

strace.log.482:03:13:07.306372 openat(AT_FDCWD, "/usr/lib/x86_64-linux-gnu/gbm/tls/nvidia-drm_gbm.so", O_RDONLY|O_CLOEXEC) = -1 ENOENT (No such file or directory)
strace.log.482:03:13:07.306389 openat(AT_FDCWD, "/usr/lib/x86_64-linux-gnu/gbm/nvidia-drm_gbm.so", O_RDONLY|O_CLOEXEC) = -1 ENOENT (No such file or directory)

So putting a nvidia-drm_gbm.so link there would make things work out of the box.

FWIW the code doing the loading is in Mesa: https://gitlab.freedesktop.org/mesa/mesa/-/blob/main/src/loader/loader.c#L620 https://gitlab.freedesktop.org/mesa/mesa/-/blob/main/src/gbm/main/backend.c#L195

[TingPing]

I believe there isn't anything this package can do. This needs to be handled in freedesktop-sdk.

They could perhaps patch libgbm, set up symlinks that point into the extensions, or even set the env var by default.

[doraskayo]

I believe there isn't anything this package can do. This needs to be handled in freedesktop-sdk.

They could perhaps patch libgbm, set up symlinks that point into the extensions, or even set the env var by default.

No need to patch. There's a gbm-backends-path Meson option: https://gitlab.freedesktop.org/mesa/mesa/-/blob/be4f7fb656180ab55a50eff01f36672b0bf5f146/meson_options.txt#L314-320

Allowing extensions to add files to the shared gbm directory also requires adding it to merge-dirs in the GL extension point.

If anyone is interested in having this functionality and is willing to test it thoroughly, an issue requesting this should be opened in https://gitlab.com/freedesktop-sdk/freedesktop-sdk/-/issues.

[doraskayo]

This should already be working in 23.08beta (freedesktop-sdk!14619) and will also be available in the next release of 22.08 (freedesktop-sdk!14640).

[guihkx]

Closing as fixed as per the comment above.