RetroArch framebuffer access limited to one

[slaminger]

Issue: RetroArch can only draw one framebuffer, this hinders certain systems that use multiple screens as they must be drawn on the same framebuffer, forcing any work around to be done by the core itself. It also has an impact on NetPlay because it shares states.

RetroArch having access to multiple framebuffers, and allowing RetroArch to dictate where and how they are drawn would improve cores such as DeSmuME and Citra. It would also allow a virtual or on screen VMU display for Flycast. And would allow RA the ability to connect to the host FB and remote FB at the same time, with the remote RA tunneling the commands to its hw, and draw the same graphics to each FB

Resolution: Implement the ability for RetroArch to draw/access multiple framebuffers, dictate as to where and how to draw the framebuffers. This option should be able to be turned on or off as it may interfere with cores that do not require it.
the core should provide default suggestion(s) where each framebuffer should be drawn by default, but it should be the job of the of RetroArch to ultimately decide where things get drawn, in case the user wants to override core defaults for example. The core will specify the number of framebuffers and their sizes/pixel formats/etc. Also assume only one refresh rate.

This means that to solve this issue, RetroArch should be able to access at least five framebuffers, and draw them where and how RetroArch would like them to be drawn. It should also have an option in the RetroArch menu to turn off multiple framebuffers, and it should not draw more framebuffers than are requested. The core may request where to draw, but RetroArch should be able to override this request and have final control. This should work with the current RetroArch API, any API bump would need to first be discussed and cleared with the RetroArch development team. It should also have at least 1 implementation, whether in an existing core or a test/proof-of-concept core

[anyputer]

It would be nice if each framebuffer had an (optional) descriptor string. (: For example, "Top Screen", "VMU Display", "Wii U GamePad"... The frontend can choose to use these to distinguish the buffers in a user-friendly way.

[ghost]

Heya, fellow enjoyers! Just wanted to check in to say that I'm working on this.

The plan is to hammer the layout system into shape, so that it could be reliably used to render multiple screens, and introduce a new libretro API extension to announce the extra framebuffers available. Players will then be notified that there's an option to enable custom layouts, the simplest of which will be built in.

This is somewhat trickier than it sounds, as the layout system wasn't updated since it was introduced and isn't very usable anymore due to world moving on. Still, I believe it to be the sensible solution, as it will require less code to write and maintain than adding an entirely new system. The downside is that the layout system is driver-dependent, and so far it is only supported for the gl2 driver. I'll update it for glcore - the most current one - but those using Vulkan or DX drivers may be left hanging.

So, what do you think? I'm open for feedback.

[slaminger]

Heya, fellow enjoyers! Just wanted to check in to say that I'm working on this.

The plan is to hammer the layout system into shape, so that it could be reliably used to render multiple screens, and introduce a new libretro API extension to announce the extra framebuffers available. Players will then be notified that there's an option to enable custom layouts, the simplest of which will be built in.

This is somewhat trickier than it sounds, as the layout system wasn't updated since it was introduced and isn't very usable anymore due to world moving on. Still, I believe it to be the sensible solution, as it will require less code to write and maintain than adding an entirely new system. The downside is that the layout system is driver-dependent, and so far it is only supported for the gl2 driver. I'll update it for glcore - the most current one - but those using Vulkan or DX drivers may be left hanging.

So, what do you think? I'm open for feedback.

If it works and makes the retroarch team happy, I'm happy.

[inactive123]

I am grateful somebody is finally working on this!

My recommendations/proposals - please design this in a way so that we can easily backport this to Vulkan/D3D11/Metal all the other big graphics APIs. Having it locked and limited to only OpenGL is a pretty bad thing, so it has to be designed in a way so that it can be trivially added to any graphics API.

I'd much prefer if you designed it in a way so that it would work on any video driver period.

Let's do this for now - come up with your Proof of Concept (targeting gl2 and glcore for now), and design it in a way so that it can easily be added for Vulkan/D3D11/Metal later, and we'll see where to go from here.

[stacyharper]

Any progress regarding this?

[slaminger]

Any progress regarding this?

It's been very slow going. A couple people have showed interest but have yet to work on it. I'm definitely still paying attention to it.