korejan / ALVR

Stream VR games from your PC to your headset via Wi-Fi
https://alvr-org.github.io/
MIT License
186 stars 27 forks source link

Alpha based passthrough #101

Open LennartJohansen opened 8 months ago

LennartJohansen commented 8 months ago

Would it be possible to add an option for alpha based passthrough to ALXR? To use as an alternative to the black color chroma key.

The streaming in Vive Business Streaming uses this for the XR Elite. It uses the alpha passed to OpenXR on the PC as source. This opens up for semi transparent areas making smooth transitions between XR and passthrough possible.

elbadcode commented 5 months ago

Reopening this issue bc I'm revisiting it myself. Vulkan seemed to have issues with decoding alpha channel when I looked into it before but I didn't try opengl es. Alpha on its own would be better than nothing but not immediately useful without preprocessing in the game renderer. There's a reshade shader that can pack depth maps into RGBA for example. That still leaves us with processing the depth map headset side and if we're already testing for depth on the PC we may as well key out the background in reshade and forego the depth map headset side. Unless we use it for additional effects like shadowmapping or to enhance accuracy of generated motion vectors used for reproduction.

I don't think I have time to pursue this but I'd like to determine if it's even possible to stream alpha to quest