When in 2D mode, use an OpenVR overlay for the browser window

[philip-lamb]

In order to make use of OpenVR protected overlays as a destination for EME media, we need to support rendering the browser texture to an OpenVR overlay (and have OpenVR composite it into the scene) instead of having Unity composite it. This will also potentially be a performance improvement.

• OpenVR overlay reference (native): https://github.com/ValveSoftware/openvr/wiki/IVROverlay_Overview
• SteamVR Unity OpenVR overlay class reference: https://valvesoftware.github.io/steamvr_unity_plugin/api/Valve.VR.SteamVR_Overlay.html

Tasks will include:

• instantiating the overlay
• positioning and sizing it in the scene (https://github.com/ValveSoftware/openvr/wiki/IVROverlay::SetOverlayTransformAbsolute, https://github.com/ValveSoftware/openvr/wiki/IVROverlay::SetOverlayWidthInMeters)
• setting it to consume the incoming DirectX11 texture (https://github.com/ValveSoftware/openvr/wiki/IVROverlay::SetOverlayTexture)
• handling mouse interaction (https://valvesoftware.github.io/steamvr_unity_plugin/api/Valve.VR.SteamVR_Overlay.html#Valve_VR_SteamVR_Overlay_ComputeIntersection_UnityEngine_Vector3_UnityEngine_Vector3_Valve_VR_SteamVR_Overlay_IntersectionResults__)
• switching the incoming texture to a Unity Texture2D target when in 360° video mode.

[patrickoshaughnessey]

I made good progress on this one this week. The current state can be found on the steamvr-overlay branch (https://github.com/MozillaReality/FirefoxRealityPC/tree/steamvr-overlay). Many thanks to @thomasmo and @kearwood for their assistance!

The current progress is:

• Unity creates an overlay using the SteamVR_Overlay class
• The browser texture coming from the plugin is assigned to the overlay
• The native laser pointer is used to interact with the window, and the overlay events are passed to the window
• The keyboard works as before
• Full-screen video does not use the overlay, but uses the existing mesh surface mechanism. Our laser pointer is only used when in full-screen mode, in order to be able to use the projection menu, as before

Known limitations:

• The overlay renders over top of everything, so if the keyboard is opened overlapping the window, the portions of the keyboard that overlap are occluded
• The "murder glove" will stay frozen in place at the point the user closed the keyboard
• Particle effects do not animate while the overlay is accepting input using the built-in overlay laser pointer

[philip-lamb]

Valves docs imply that it ought to be possible to get correct depth sorting, but I'm not sure of the exact mechanism. I wonder if we there is a way to deal with the occlusion problem via a "two-pass plus scissor"-technique where the overlay is drawn first, and then the immersive scene second, with the areas of the overlay that aren't occluded by the hands/laserpointer etc left undrawn. This would suggest that we'd have to have the immersive scene render onto a transparent frame buffer.