[libretro] [Linux] Wayland or X required for Vulkan

[konistehrad]

What happens?

Currently the only way to display a Vulkan context on Linux is via X or Wayland. If a KMS session is started, the application will fail to launch. This is particularly notable when using the libretro core, as Retroarch has full support for Vulkan in KMS.

What should happen?

The application should start, using vkCreateDisplayPlaneSurfaceKHR to configure a full screen display.

What hardware, operating system, and PPSSPP version? On desktop, GPU matters for graphical issues.

Rpi4 on Ubuntu 20.10 is the test target.

I've started work in a branch on resolving this issue, but I'll concede immediately I don't have a ton of experience writing Vulkan code, and most of what's done is the boilerplate extension loading and validation. I'm using retroarch's khr_display_ctx.c as a guideline.

[orbea]

Duplicate issue. https://github.com/hrydgard/ppsspp/issues/13135

[konistehrad]

Sorry, looked for Vulkan, not KMS. Thanks for marking this!

[konistehrad]

Upon further review, I'm not sure this should have been closed; it should be possible to start a vulkan session without using SDL2 as a context. Many of the other libretro cores support this display mode, and I think PPSSPP should have parity with those. Please let me know if I'm right about this and should reopen the ticket. Thanks!

[orbea]

@konistehrad I am not a definitive source on this, but my understanding is that ppsspp standalone will require SDL2 regardless of the front end. Its used for much more than just the graphics. So if this will work it will likely require SDL2 to also support the vulkan KHR context. They at least support a KMS context which could be used with opengl and opengles. I'm not sure if Qt5 could be used this way, but that is an even heavier dependency.

[hrydgard]

Well, initializing a VK KHR-surface context doesn't require any cooperation from SDL, does it - even if we are using SDL for input etc. I think?

[orbea]

Well, initializing a VK KHR-surface context doesn't require any cooperation from SDL

I'm not sure if how much of the UI would work without SDL2 or Qt5?

[hrydgard]

all of our in-window UI is drawn with GL or VK. In the SDL port that's all the UI we have. Basically to initialize VK for X all we need is a window handle, which we do get from SDL.

Qt port has some actual windowed UI and a window menu so that port is a bit different.

[orbea]

Then It sounds plausible that both the VK KHR-surface and KMS contexts could work without SDL2. I am not sure if it would be preferable to leverage any existing SDL2 support or to implement it entirely in ppsspp? I would defer to your judgement if supporting the VK KHR-surface context is the same or a different issue than why the SDL2 KMS context doesn't work with ppsspp when using opengl.

For the end user the basic difference is if its using gl or vulkan and I don't think it would matter how its accomplished as long as it works.

[konistehrad]

I actually can confirm that my branch works on an Rpi4, no SDL, no Wayland, no X, via retroarch but only if you disable the VK_KHR_sampler_mirror_clamp_to_edge extension on libretro_vulkan.cpp:110 (Rpi4 doesn't seem to support this extension at this stage.) Native ppsspp modesetting would require additional logic as vkGetInstanceProcAddr_libretro "eats" the call to vkCreateDisplayPlaneSurfaceKHR, but the code paths do all seem to check out. (Edit aside: is VK_KHR_sampler_mirror_clamp_to_edge actually needed here?)

[orbea]

How about without retroarch? Libretro is too broken to use.

[konistehrad]

I'll have to extend ppsspp to examine the screen to get it to work. I'll start cracking on a standalone solution this weekend. Also thanks for the note on the libretro core, it had been working pretty well for me, but I'll focus on standalone for now.

[konistehrad]

OK I have a much clearer picture of the different build modes now, thanks for y'all's patience. I'm restricting this to libretro, as SDL seems required otherwise according to CMakeLists.txt.

[orbea]

Does the KMS/Vulkan KHR work with any other gl/vulkan libretro cores? Are you sure this is not a RetroArch issue? It would not be surprising if they broke it on their end. At the very least this all worked years ago before they started putting profit before quality.

[konistehrad]

Yep, it works super well with their Beetle PSX HW emulator fork, as well as their Beetle SNES fork. I wanted to try the N64 version as well, but I haven't gotten around to it yet.

[orbea]

Then I would suspect in the case of libretro its a regression introduced in the ppsspp repo, Probably standalone ppsspp was improved while the necessary changes for libretro were never done.

[gouchi]

@konistehrad it seems VK_KHR_sampler_mirror_clamp_to_edge has been added few days ago for v3dv.

Source

[amverweij]

I can confirm that after applying https://github.com/konistehrad/ppsspp/commit/e3540ccd17193e131d42290a9dd4d6bca6e5f634.patch to current git master (4f6ae0b322d7dd12d279fc1ee80c3c1b9611f2c5), when using mesa 21.2.0, the libretro core of ppsspp works on a RPi4 with the vulkan driver in a KMS/DRM session. (Without the patch, it just crashes.)

ppsspp_libretro.so was compiled using the following commandline:

CMAKE_ARGS="-DUSING_GLES2=ON -DUSE_FFMPEG=ON -DUSE_SYSTEM_FFMPEG=ON -DUSE_SYSTEM_LIBZIP=ON -DVULKAN=ON -DUSE_DISCORD=OFF -DUSE_MINIUPNPC=OFF -DUSE_VULKAN_DISPLAY_KHR=ON -DUSING_X11_VULKAN=OFF" ./b.sh --rpi64 --libretro --release

[hrydgard]

Looks mergable - wanna make a PR?

[gouchi]

We have tested the patch on Lakka and it is working also on RPi4 using v1.11-hotfixes branch.

[gouchi]

@konistehrad So we can can close this issue as PR has been merged.

Thank you.