Shaky and stuttery head tracking with 100+ boxes on Android

[7ammer]

On my phone (nexus 6x), as I add more geometry to my scene I've noticed that head tracking becomes slow, stuttery or shaky. However using touch to look around is as smooth as butter. This seems a bit odd to me.

I've created a scene to help explain the issue: https://jsfiddle.net/xkzbwh3u/ On my phone when head tracking: 001 - 100 boxes = is fine 100 - 200 boxes = some noticeable shakiness after sudden movements and slight stuttering 500- 1000 boxes = very shaky, stuttery and sluggish

As explained above, looking around by simply touching or swiping the screen shows no shakiness at any of these ranges, at least not from what I can tell.

It looks like, to me, that Android's accelerometer is finding it harder and harder to provide data to the browser as I fill up the scene with objects.

Is there anything that can be done about this?

[ngokevin]

Android throttles or provides high-latency access to the accelerometer/gyroscope sensors. This is probably pronounced as performance drops. If you are using lots of boxes, there are techniques to improve performance:

The performance doc mentions geometry merging

Or use three.js geometry instancing (there are some WIP components by @takahirox to enable this): https://threejs.org/examples/webgl_buffergeometry_instancing.html

[7ammer]

Hi @ngokevin, thanks for your performance suggestions. Do you have any advice for collada models? Can you mergeTo a collada model?

[MilllerTime]

I'm experiencing the same problem on Android (7.0, OnePlus 3T). I realize this issue doesn't lie solely with A-Frame, but I think it's an important one as new VR creators try to get up and running quickly and want to show off their creations on whatever hardware they have.

@ngokevin A couple of questions if you don't mind:

• Do you have links to any resources about this issue at the Android level? I'd be curious why it exists and if there is any effort to resolve it.
• Is it possible that the WebVR spec will eliminate this "sensor lag" for mobile devices such as Android?
• Are there any possible non-graphics workarounds, such as smoothing the sensor input across a couple frames? I ask because I'm curious if this would help, and I assume it would need to happen within the look-controls component. I haven't found any code to suggest this is being done.

I just started tinkering with A-Frame, and built my first VR experience. I'm extremely impressed how it works with tried and true web technologies, and how low the barrier to entry is. That being said, this issue increases the barrier to entry for anyone using Android. I do have several hundred boxes in my scene, but my phone easily renders it at 60+ fps. There's no apparent performance issue, though the head tracking is still jittery. While my demo served the purpose of learning about A-Frame, it's difficult to show others with all the stutter.

The boxes in my scene are all colored differently (based on some math), so merging the geometry would mean rewriting the entire demo to use a shader, texture, or other means of achieving the same image. Given that modern hardware has no problem rendering this scene as is, it's hard to find motivation/time to rewrite this toy just so Android sensor readings may be more performant. Further, anyone who is new to 3D may be able to populate a scene with primitives, but needing to aggressively tune the performance of such simple scenes could easily be a turn-off.

I appreciate any information on this!

[ngokevin]

Unfortunately, we should lower expectations on what non-WebVR enabled devices can provide. With non-WebVR devices, we are literally polyfilling proper WebVR head pose data using Device Motion API. Official WebVR-enabled devices provide data directly and optimized through the WebVR API proper.

While many Android devices today are OK for tinkering for newcomers to at least play with, many of them will exhibit non-smooth tracking (as seen since you have 60FPS but bad jitters).

The code relevant is at https://github.com/googlevr/webvr-polyfill

[MilllerTime]

Thank you for the quick response. This definitely makes sense.

Given that this won't be an issue with proper WebVR support, I can understand setting lower expectations for polyfilled experiences. Thanks again 🙂

[derhuebiii]

Worth to mention that callibrating the sensor on my Galaxy S7 just did wonders. I thought it was laggy fps-wise but then I figured it was running quite smoothly without the gyrosensor.

Callibration on S7 is making this "8" with the phone, Google has a good explanation here: https://support.google.com/maps/answer/6145351?hl=de