RmlUi GLFW3 (Vulkan) causes amgpu driver crash

[niansa]

On Debian Trixie with an AMD RX 5700 XT, the following code snippet causes a crash of the amdgpu driver when running with the GLFW3 Vulkan backend/renderer:

if (!Backend::Initialize("Title", 1920, 1080, false)) {
    return;
}

Rml::SetSystemInterface(Backend::GetSystemInterface());
Rml::SetRenderInterface(Backend::GetRenderInterface());

Rml::Initialise();

auto ctx = Rml::CreateContext("main", Rml::Vector2i(1920, 1080));

Rml::LoadFontFace("documents/LatoLatin-Regular.ttf");
auto doc = ctx->LoadDocument("documents/Home/index.rml");
doc->Show();

while (true) {
    Backend::ProcessEvents(ctx);
    doc->UpdateDocument();
    ctx->Update();
    Backend::BeginFrame();
    ctx->Render();
    Backend::PresentFrame();
}

[wh1t3lord]

Can you compile in debug with RMLUI_VK_DEBUG (uncomment preprocessor)? Mostly we tested on nvidia videocards I don't know what could be wrong with AMD. https://github.com/mikke89/RmlUi/blob/master/Backends/RmlUi_Renderer_VK.h#L83C2-L83C27

@niansa

And which version of RmlUI do you use?

[mikke89]

Ouch, this pattern looks a lot like a dying GPU :/ You don't happen to be overclocking it? I'm sure it could also be a driver issue, and perhaps in combination with something we do. Enabling debug output and validation layers (as mentioned above) would be very helpful.

Did you try any other backends?

[niansa]

I'm on version 23b93483c576776c04f8eb7d221b0b0b4ad392bd. I have also tried the GLFW GL3 backend, but only got a black window with that. I will get you more info as requested later when I'm on my PC.

[niansa]

Here's a debug log:

Loaded font face 'LatoLatin' [regular] from 'documents/LatoLatin-Regular.ttf'.
Rml width: 3840 height: 2160 | Vulkan width: 3840 height: 2160
can't be, you must have a valid count that bounds from minImageCount to maxImageCount! Otherwise you will get a validation error that specifies that you created a swapchain with invalid image count
/home/nils/Programs/OSS/QSysLib/RmlUi/Backends/RmlUi_Renderer_VK.cpp:1782
can't be, you must have a valid count that bounds from minImageCount to maxImageCount! Otherwise you will get a validation error that specifies that you created a swapchain with invalid image count
/home/nils/Programs/OSS/QSysLib/RmlUi/Backends/RmlUi_Renderer_VK.cpp:1785

can you test on current master branch please?

Already been on latest master.

[niansa]

Oops

Also found this issue: In RmlUi_Renderer_VK.cpp

[wh1t3lord]

@niansa did you have VK_ICD_FILENAMES= thing?

As you understand I don't have your system and I don't have AMD videocard and probably you should debug by your own tbh. Googling says that it might be https://stackoverflow.com/questions/36486036/vulkan-on-x11-vkgetphysicaldevicesurfacecapabilitieskhr-error#36489432

First of all you can try to run vulkaninfo and vkcube (it is 3d sample out-of-box if you installed vulkan sdk on linux) and if it works fine then it means that your OS operates bad (I mean GLFW works bad on your OS) or drivers, because specs state that returning value can't be less than 1 but in your case it is 0.

Also can you show what GetSurfaceCapabilities return for you?

[wh1t3lord]

@niansa you need to paste all information what vulkaninfo writes. It is important to understand on which side the problem, because if vulkaninfo doesn't print a valid information about your videocard that means it is driver's problem and you need to choose different version of drivers or just reinstall them.

[niansa]

Okay, here's something interesting. I had libglfw3-wayland installed instead of the normal X11 package. Installing the X11 package fixed the issue! Is there a known problem with GLFW3 wayland?

vulkaninfo.txt

[wh1t3lord]

@niansa I dont know, for me it is strange but I am not Linux owner. I could test it under wsl, but if you fixed your problem then Wayland version is not good for you. ¯_(ツ)_/¯

Also your vulkaninfo is valid and all is fine.

[mikke89]

@niansa Which version of GLFW are you using? The most recent one (3.4) has several changes and fixes for Wayland in particular. Could you try that? Also would be interesting if you could test the SDL_VK backend with Wayland to see if that makes a difference.

[niansa]

Debian Trixie provides version 3.3.10, so that's what I'm using. SDL Vulkan seems to be running with Wayland right out of the box and with no apparent issues.

[wh1t3lord]

Any reason for keeping this issue as opened?

[mikke89]

@niansa I would encourage you to test this on GLFW 3.4 and report back. In the meantime, it seems likely to be an issue outside the library, so I am closing this one.