search

FusedVR / VRStreaming

Unity Render Streaming SDK to stream VR from CloudXR to WebXR over WebRTC
Apache License 2.0
147 stars 24 forks source link

Late Latch Functionality on WebXR Client #26

Open vmohan7 opened 3 years ago

vmohan7 commented 3 years ago

To improve experienced latency, we would want to adapt the video streams for the WebXR Client to use some sort of late latch technology like asynchronous timewarp to adjust the frames based on the latest input from the user.

Greg from Nvidia does a good job explaining the value proposition for this in the CloudXR AWE session : https://youtu.be/hYqzFdLleHI?t=1974

vmohan7 commented 3 years ago

Needs more research, but this may help : https://aframe.io/docs/1.2.0/components/layer.html

EDIT : in the right direction; however, this is primarily meant for 360 video and takes up whole field of view

vmohan7 commented 2 years ago

After some digging, looks like at the moment, the Layers API is Oculus Browser only : https://discordapp.com/channels/758943204715659264/759166287901098024/899028332442681385

Moreover, this may not necessarily be the right long term solution as without a depth map, async timewarp won't be as effective to align content with the view.

vmohan7 commented 2 years ago

Seems like this might be an option to support multiple browsers as a short term fix : https://github.com/immersive-web/webxr-layers-polyfill

I do think depth maps would be the best way long term but seems like no Nvidia CloudXR supports Depth Maps for async timewarp