Lowering latency / eliminating drift - how ?

[motorsep]

I figured that the only things that make Cardboard experience not so great are latency and drift.

What are the ways to eliminate drift and lower latency ?

Is it more of a hardware issue ? If so, what smartphones have the fastest sensors to provide least latency when using Cardboard VR ?

[hannibalov]

Hi, I'm also highly interested in this thread. Samsung/Oculus have developed a lot of optimizations such as time warp and late latching, described in their blog. I remember I read somewhere that they also introduced a lot of optimizations for GearVR on a low level such as direct front buffer rendering (http://www.engadget.com/2014/09/04/samsung-gear-vr-john-carmack/) and an android kernel priority interruption layer for sensors (or something similar, can't remember where I read that).

It would be nice to know which of these optimizations, if any, are built in the Carboard plugins. Maybe there's documentation I haven't found?

Although I can't tell for sure, I would think that sensors on mid to high-end phones aren't that bad. In fact I recall reading somewhere that the sensors on the gearVR are only streaming data 2X as fast, but precision-wise are similar (please correct me if I'm wrong). Despite that, the feeling of responsiveness is just great on gearVR compared to a similar app for cardboard. Given my relatively little experience, I'm inclined to think that latency and drift are mainly improvable by software. That doesn't mean it's an easy task, by no means, and probably with a lot of compromises given the diversity of android devices. But hey, one can dream :)

As I said, correct me if I'm wrong in any of the points

[smdol]

Cardboard is just an SDK that runs on top of whatever version of Android you have on whatever phone you have. If this whole VR thing really takes off, I expect there will be changes similar to what Samsung did for Gear. You're right it is mostly a software problem, but some crucial parts of it are at the OS level.

As far as good current hardware, I've found that iPhones rank up there (personal opinion). In fact, the SDK turns off drift correction for iOS because it is basically not needed.

[motorsep]

Hmm.. So is there no way you guys can add some hack to SDK to workaround phone's OS to reduce latency ?

It seems that if everyone waits on one another, to make it happen, it would lead to VR not taking off :) So far the Cardboard is the only "device" that 1M+ people have on hands. The rest of the devices can't even get close to 1/2M units (I've heard that Gear VR managed to ship less than 200k units to the date).

I tried iPhone 5S and the screen on that thing is just horrible. I haven't tried anything moving to test latency, but viewing photospheres on iPhone 5S was such a turn off for me.

[smdol]

I use an iPhone 6 for testing. It's much better.

[motorsep]

So in other words, you can't implement any hacks to Unity SDK to reduce latency on Android, correct ?

[smdol]

Several hacks are in the native code, but we may be at the limit of what can be done on current stock Android. The other tack is to optimize the app itself.

For comparison, have you looked at the Cardboard Design Lab app? It is the best performing Cardboard app I know of. How does latency look with it?

[motorsep]

I haven't yet. I will check it out for sure. Can you retrofit their optimizations to Unity SDK ?

[smdol]

Yep, they used a modified version of the SDK to do distortion correction in vertex shaders. Works really well, but it's a bit more work on the app developer's side.

[motorsep]

@smdol I tried Design Lab and it does seem to be more responsive than stock Unity SDK scene and other apps I tried. However I think my Cardboard itself is bad - I get horrible eye strain on my right eye when using it. The issue might be stemming from the adjustable IPD on my Cardboard. I am waiting for the normal one, with fixed IPD, to arrive and then I'll test again, but either way it would be nice to port all the optimization from Design Lab app to Unity SDK.

[motorsep]

@smdol So I finally got my Knox V2 viewer - no eye strain, better quality of lenses.. Unofficial Cardboard viewer was all wrong :/

So I tried 2 things - Design Lab and viewing my JPGs photospheres ( https://drive.google.com/drive/folders/0BwE6dxM0O2PsakNBV0lURV8xSGs ) using Galaxy S3 and Galaxy S6 Edge.

Design Labs was really awesome. I liked the responsiveness even on S3. It would be really nice to port all their goodness into Unity SDK. Although adding Sensor Fusion app tracking to the mix would be even better.

I had pretty bad experience with photospheres. It doesn't have to do anything with Unity, but I don't know where to report it. The updating of the image as I was looking around was super choppy, even on S6 Edge. I don't quote recall what it was when I used uncompressed PNGs, but I don't think it was that choppy. And on Gear VR nothing was choppy, neither with PNG nor JPG.

[nathanmartz]

The CDL scenes use single pass distortion, which requires shader modification but we include suggestions in the CardboardDistortion.cginc file bundled in the SDK. We also added Unity native rendering in 0.5.2 which is somewhat faster than the old distortion correction and we fixed a regression in 0.6.0. Not much more we can do here, so closing.

[andregm3]

i got a lot of fps by just disabling distortion correction:

i think the distortion should came right from the camera in unity, not by post-processing.

Btw...i'm using aspherical 37mm lens with 50mm FL so i'm really not needing distortion correction with them the screen is not getting distorced.

[smdol]

That's not really recommended unless you are going to implement vertex-displacement distortion correction.

http://gamasutra.com/blogs/BrianKehrer/20160125/264161/VR_Distortion_Correction_using_Vertex_Displacement.php

[andregm3]

Also editing the Cardboard(script) and turning the line to (true):

private bool electronicDisplayStabilization = true;