vulkan: vkAllocateDescriptorSets failed | 6700XT + Mesa 24.1 on Gentoo ~amd64

[CNR0706]

Hi there, when trying to launch gamescope in nested or embedded mode I am greeted with these two error messages: vulkan: vkAllocateDescriptorSets failed and SDL_Vulkan_CreateSurface failed: VK_KHR_wayland_surface extension is not enabled in the Vulkan instance.terminate called without an active exception

My system has a 6700XT running on Gentoo ~amd64. My Vulkan drivers are: Mesa 24.1, AMDVLK 2024.2.1, AMDGPU-Pro 23.40.0.1718238 and lavapipe. All of them have multi-lib support.

Relevant env vars I set are: DISABLE_LAYER_AMD_SWITCHABLE_GRAPHICS_1=1 and VK_ICD_FILENAMES='/usr/share/vulkan/icd.d/radeon_icd.x86_64.json:/usr/share/vulkan/icd.d/radeon_icd.i686.json'

All Vulkan applications, native or not, work in this setup. It's only gamescope that's giving me trouble.

--- Terminal output ---

~
❯ gamescope -- glxgears
ATTENTION: default value of option vk_khr_present_wait overridden by environment.
ATTENTION: default value of option allow_draw_out_of_order overridden by environment.
vulkan: selecting physical device 'AMD Radeon RX 6700 XT': queue family 1 (general queue family 0)
vulkan: physical device supports DRM format modifiers
vulkan: vkAllocateDescriptorSets failed
SDL_Vulkan_CreateSurface failed: VK_KHR_wayland_surface extension is not enabled in the Vulkan instance.terminate called without an active exception
[1]    20303 IOT instruction  gamescope -- glxgears

~
❯

--- System Information --- https://gist.github.com/CNR0706/3dfa798873ac2ca6f37c14637ad4d098

--- System Report --- https://gist.github.com/CNR0706/8e4b258eef17a6c8e77d30d650e3d769

--- Gamescope version and USE flags --- gui-wm/gamescope 3.14.14 gui-wm/gamescope pipewire wsi-layer filecaps

[kisak-valve]

Hello @CNR0706, blind guess, since this is an SDL error message, double check that media-libs/libsdl2 is built with USE="vulkan wayland".

Also, there's a known AMDGPU regression in the 6.8.9 kernel. Using a different kernel version would increase your chances of success.

[CNR0706]

Explicitly enabling "vulkan" and "wayland" didn't help unfortunately. I also switched to the 6.9.x Gentoo Kernel but it also didn't make a difference.

[CNR0706]

I'm a little confused about the message: "vulkan: selecting physical device 'AMD Radeon RX 6700 XT' ... "

Shouldn't it say "AMD Radeon RX 6700 XT (RADV)"? If it's using AMDGPU-Pro or AMDVLK, that would explain why it doesn't work.

[Jozhin-s-Bazhin]

Hi, I'm getting the exact same issue on NixOS with a 7700S and Mesa 24.0.7

Edit: I'm using Hyprland as my DE i that's relevant

[CNR0706]

Hi, I'm getting the exact same issue on NixOS with a 7700S and Mesa 24.0.7

Edit: I'm using Hyprland as my DE i that's relevant

Do you have any alternative Vulkan drivers installed? (AMDVLK, AMDGPU-Pro, lavapipe...)

[Jozhin-s-Bazhin]

Turns out I do have AMDVLK installed. After disabling it gamescope works. Thank you

[CNR0706]

Turns out I do have AMDVLK installed. After disabling it gamescope works. Thank you

Was AMDVLK set as your default driver or did you override it like I did using VK_ICD_FILENAMES?

[Jozhin-s-Bazhin]

I didn't override it like you did, but it wasn't the default. This stuff is abstracted on NixOS, so I have no idea what is going on under the hood.

[ParetoOptimalDev]

I didn't override it like you did, but it wasn't the default. This stuff is abstracted on NixOS, so I have no idea what is going on under the hood.

It says:

The AMDVLK drivers can be used in addition to the Mesa RADV drivers. The program will choose which one to use:

from https://nixos.wiki/wiki/AMD_GPU#AMDVLK

More info: https://nixos.org/manual/nixos/unstable/index.html#sec-gpu-accel-vulkan

[ParetoOptimalDev]

Turns out I do have AMDVLK installed. After disabling it gamescope works. Thank you

How did you disable amdvlk on NixOS exactly?

[Jozhin-s-Bazhin]

Adding the snippet you're referencing makes amdvlk the default driver. I simply commented out that part making radv the default again. Here is a snippet from my config:

  services.xserver.videoDrivers = [ "amd" ];
  ...
  boot.initrd.kernelModules = [ "amdgpu" ];
  hardware.opengl = {
    enable = true;
    driSupport = true;
    driSupport32Bit = true;
    extraPackages = with pkgs; [ 
      #amdvlk 
      #driversi686Linux.amdvlk
      rocmPackages.clr.icd
   ];
  };