IPC implementation for ReShade FX runtime uniform variables

[wheaney]

I have a branch of vkBasalt where, a few months ago, I built out a solution for passing runtime uniform variables without needing a complex GUI solution like ReShade provides nor needing to build a C++ plugin that does memory copies like ReShade would otherwise require for dynamically set values. My implementation does this by allowing communication with the Vulkan layer via inter-process communication (or IPC, specifically in my case, shared memory). The benefit of allowing uniforms to be passed via IPC is that a shader can be written in ReShade FX and uniform values can be passed to the shader using whatever technology the developer pleases, they just need to conform to the IPC protocol chosen.

My runtime uniform implementation can be seen here (everything but the Docker build stuff).

An IPC solution would be amazing in Gamescope because the GUI for configuring a shader's settings can be built in any application/framework, or even be baked into Gamescope itself as a UI in sidebar/Steam menus. It can also allow for more dynamically controlled shaders like the one I've built that takes IMU data from an XR headset and transforms the game image based on where the user is looking.

Since I built my solution on vkBasalt itself, and Gamescope's reshade uniforms implementation appears to be mostly taken directly from vkBasalt, merging in my implementation of this would actually be pretty straightforward. If by some miracle the Gamescope gods think this might be a useful solution, I'd be happy to create a PR.

[wheaney]

I forgot to mention that my implementation is actively used by a few thousand people, mostly on Steam Deck, so it's well tested and proven.

Here's a working example of a shader defining uniforms that are dynamically set by another process via IPC.

I have an application that reads data from an XR headset IMU and writes it to IPC for the shader to consume. Here's where it sets up IPC and here is where it writes those values. Since it's shared memory, it's fast, picked up immediately on the next frame for shader rendering, and no disk IO is needed.

[wheaney]

As a sort of proof-of-concept of my statement that the shader settings GUI can be implemented in any framework and able to live anywhere, my plugin UI illustrates this well in the Steam Deck sidebar, since it's just a React web UI and some of these values are just simple configurations that are passed directly to the shader from a UI element that the user controls. The display size slider seen in this screenshot is one example, it ultimately writes directly to this uniform variable via IPC.

[wheaney]

@Joshua-Ashton Any thoughts on this one? I could work on a PR but that could be a lot of effort for something you'd reject on principle, so I'd like to gauge your interest before I do that.

[pravic]

@wheaney Amazing work!

It appears that some of your links are broken now:

• https://github.com/wheaney/xrealAirLinuxDriver/blob/main/src/virtual_display_plugin.c#L51-L68 - 404 for the file
• https://github.com/wheaney/sombrero/blob/main/IMUAdjust.fx#L13-L35 - shifted after an update

To prevent this, it's better to copy links as "permalink" - it will point to a specific revision rather than the master which is volatile.

[wheaney]

Oh shoot, you're right. I've updated the links. Thanks!

[misyltoad]

You should just use a Wayland protocol in gamescope-control to control it instead of your own ipc.

[wheaney]

Can you elaborate on that? You're suggesting that I expand on the gamescope-control wayland interface to provide a generic way to send data to reshade shaders?

[misyltoad]

Yes, add the stuff there

[wheaney]

So, without being very deep on Wayland, the little I do understand raises a couple questions/concerns for my specific use case:

• I'm reading IMU data at a rate of at least 250Hz and sending that immediately to my shared memory for IPC. For a game rendering at 60 frames-per-sec, most of those get ignored but it doesn't really matter because I'm just writing to memory. With Wayland I'd be spamming that over the protocol to be handled by the request queue.
• Since I need my data to be as real-time as possible for a smooth experience, would the Wayland request handling be responsive enough, or would it add latency in the worst case? Any amount of added latency will result in a more choppy/jittery experience.
• For gamescope-control specifically, there's this warning: "This is a private Gamescope protocol. Regular Wayland clients must not use it." This particular addition wouldn't really be intended to be for private, gamescope-only usage.

[misyltoad]

Steam uses the Gamescope Control protocol. Feel free to make a Gamescope Shader protocol or whatever.

I think sending data over the Wayland socket should be fine for your usecase. Games can render faster than 250Hz.

[wheaney]

@Joshua-Ashton I've finally come back around to this and implementing a Wayland protocol as you recommended. I haven't tested it yet as I need to flesh out my client now, but I've laid the groundwork and everything builds and runs, and I wanted to get some early feedback before I invest too much more time into this approach: https://github.com/ValveSoftware/gamescope/compare/f554d886093ad3fdc361b4642fe6cca4cdaa99b1...wheaney:gamescope:master I've tested it from my client now and verified it works as intended.

[wheaney]

PR was merged. Closing this out.