Vulkan apps default to my secondary GPU, performance issue

[Brisse89]

Hi! Not sure if this is the right place. I was redirected here after contacting Feral Interactive support. I'll just copy/paste what I wrote to them. I think it will explain the issue perfectly well.

Hi Feral! This isn't really a support request. It's more of a suggestion for improving performance on multi-GPU systems. I'm on Debian Sid with kernel 4.15 and Mesa 17.3.9 and noticed that Vulkan games, including Rise of the Tomb Raider runs on my secondary GPU. By switching over to my primary GPU in the Feral launcher, I've increased my benchmark score from 60fps to 73fps (1080p high) and the game runs noticeably smoother. This is something I have to do every time I launch the game because the setting resets between play sessions. It's a bit weird that the primary GPU is in the bottom of the list. I wonder if this is a RADV issue since it happens in all Vulkan applications, including non Feral apps.

Debian Sid Kernel 4.15 Mesa 17.3.9 Ryzen 1700X Asus PRIME X370-PRO 32GiB DDR4-2400 cl14 Sapphire Nitro R9 Fury (two of them)

[krOoze]

Vulkan API does not really prescribe any specific order of the physical devices. The games and apps should simply be a bit more smart advising the better option to users — and, well, remember the choice.

I bet the order is simply how they are found in the discovery folders.

[Brisse89]

This is what Feral support said about how they choose GPU

We choose GPUs based on the one with the most VRAM, otherwise they're ordered by the order Vulkan returns them in. We can't know which of these is "Primary", so we assume Vulkan gives us the first one.

In this case, both of the GPU's are identical, with the same amount of VRAM. I'd also like to point out that OpenGL applications do not have this issue and picks the primary GPU by themselves.

[leigh123linux]

Vulkan uses the first working config loaded, it's the program/'driver vendors'/distro to choose which is used.

renaming the vendor provided amd driver file will make it load first

example:

su
cd /usr/share/vulkan/icd.d
mv radeon_icd.x86_64.json amd_radeon_icd.x86_64.json

Seems like a dumb choice for distros to package the mesa driver configs in one package

https://packages.debian.org/sid/amd64/mesa-vulkan-drivers/filelist

$ rpm -qf intel_icd.x86_64.json radeon_icd.x86_64.json
mesa-vulkan-drivers-18.0.1-1.fc28.x86_64
mesa-vulkan-drivers-18.0.1-1.fc28.x86_64

[Brisse89]

@leigh123linux I think you misunderstood. This isn't a case of different vendors, such as if I had an Intel iGPU (which I don't) and additional discreet graphics card. This is about having multiple discreet graphics cards of the same vendor and the same type, and for some reason, Vulkan runs on the headless card, which it shouldn't.

I guess I should bring this up with Mesa developers instead?

[krOoze]

@Brisse89 I don't think that would help. As I said Vulkan does not prescribe any particular order. Vulkan does not even assume you will draw to a monitor, so it does not even necessarily make sense to order GPUs the way you propose. And such change would be only non-specified behavior of Mesa driver, which is road to hell.

On the other hand, OS platfom APIs are notoriously messy and I do have trouble finding any decent example how to handle this.

May I suggest you bring this to Vulkan-Ecosystem. They should decide what is the best way forward(, which may be specifying the loader should order the GPUs on specific platform, or maybe Vulkan extension to detect the primary monitor connected GPU, or maybe providing readily available code sample for developers how to handle the case).

[Brisse89]

Mesa bug report here: https://bugs.freedesktop.org/show_bug.cgi?id=106187

[mdiluz]

Just to say here - as in the freedesktop report - the failure to remember the selected GPU is an app issue and will be resolved there.

I don't have a fixed opinion on whose role it is to ensure the right GPU is default here though, @krOoze maybe be right that this is really an ecosystem question as it does touch quite a few parts of the pipeline.

[karl-lunarg]

@Brisse89 Thanks for raising this issue and running it past the Mesa folks. As others have said, the Vulkan loader is operating within the specifications here. In this particular case, there is one driver reporting two physical devices and the driver is deciding the order of the devices in the list it returns. One could argue that the driver could figure out which physical device is primary if having a monitor plugged in is the necessary criteria for being primary. But this is just a soft heuristic and may not cover all cases, such as both cards having monitors plugged in.

The Vulkan loader isn't given enough information to determine the "primary-ness" or "secondary-ness" of the physical devices.

Since the "primary" attribute of a device might be determined by a windowing system - whether or not the device is plugged into a monitor, or is designated as primary in a multiple monitor desktop, it seems that some sort of additional Vulkan WSI (Window System Integration (extensions)) functionality may be needed.

As @krOoze suggests, it may be best to present this to the ecosystem group. Another approach would be to go to the Vulkan Working Group directly. Either group might defer to the other, but I think I'd suggest starting with the Working Group, as they may already have some thoughts on the subject.

It would be great if you could open another issue in either of those two places and then close this one.

Thanks!

[krOoze]

@karl-lunarg I am just gonna be a jerk, and go ahead and create one...

[krOoze]

@Brisse89 Ups, sorry. I will merge my Issue with yours then...

[karl-lunarg]

Thanks!