Video latency minimization

alvr-org / ALVR

Stream VR games from your PC to your headset via Wi-Fi

MIT License

5.26k stars 473 forks source link

Video latency minimization #251

Open zarik5 opened 4 years ago

zarik5 commented 4 years ago

Video latency minimization with a statistical model. The same model can be used in multiple ways to mitigate latency and jitter. It does not require time sync between server and client because it self-corrects.

[ ] Create EventTiming
[ ] Apply model for total pipeline latency reduction (reduces the queue time. This is probably already handled by Oculus runtime)
[ ] Apply model for black pull reduction and total latency estimation
[ ] Apply model for headset jitter. Controller jitter cannot be reduced due to a limitation of the OpenVR driver API.

fangguanya commented 3 years ago

@zarik5 Hi, Could you please explain further about why was controller jitter cannot reduced for me? Thanks. I didn't know about the difference between HMD tracking with Controller tracking well.

zarik5 commented 3 years ago

Maybe it's possible to have more control over the controller jitter if we disable the OpenVR controller prediction algorithm. I'll see what I can do when I get to that point.

fangguanya commented 3 years ago

Maybe it's possible to have more control over the controller jitter if we disable the OpenVR controller prediction algorithm. I'll see what I can do when I get to that point.

Cann't we just use the same way that we treat with sync of HMD position?

zarik5 commented 3 years ago

No, because OpenVR frame timing information contains info only for the display reprojection. If we let SteamVR do its controller prediction then we don't have correspondence between video and tracking frame anymore, thus we get judder. Anyway, I don't have everything fresh in mind, I'll see what I can do in the next major releases.

fangguanya commented 3 years ago

Great!