Use Sensor APIs when available instead of devicemotion. Fixes #10

[jsantell]

Initial pass at using generic sensors API. Apologies for the extra noise in this diff -- the related areas are start(), onGyroscopeRead_(), onAccelerometerRead_() and onSensorError_().

Right now it works on Chrome M63 with flags enabled, but even though rendering at 60FPS, the actual pose orientation is very choppy, literally maybe 1-2 frames a second. Think it may have to do with the fusion and pose estimation, but not terribly familiar with that area.

Outstanding questions:

• How will the iframe scenario work? Would it be up to developers to set the appropriate permissions on the iframes to allow sensor API to work? In this case, I think we are covered, as we would want it to fail and not fallback to device motion, I think, without appropriate permissions.
• More of a generic webvr-polyfill question, but how can we alert a user that there's a failure and there's nothing we can do to support this device, after creating a VRDisplay? In some cases, it could be solved with permission changes.
• A little stumped on the choppy fusion sensor, any ideas?

cc @cvan

[kenchris]

Wouldn't it make more sense to use the AbsoluteOrientationSensor (or Relative*) as then you don't need to do manual fusion as it happens on the sensor hub on most phones

[kenchris]

Also if you just want to do a complementary filter, that is quite simple with Generic Sensors, like

https://github.com/intel/websensor-compass/blob/master/scripts/compass.js#L301

[jsantell]

Fixed the choppy issue so it works with the complimentary filter, as well as increased the frequency to 16.6 on both sensors; pushed changed.

@kenchris (cc @cvan) I haven't looked at OrientationSensors yet, doing so now! Would this make the complementary filter redundant? Ultimately we just want a quaternion on every frame, but not sure of the pros/cons of OrientationSensors vs complementary filters with Gyro/Accel. If a browser supports Gyro/Accel, will OrientationSensors also be supported, and vice versa, or will one be more ubiquitous?

[jsantell]

From the RelativeOrientationSensor:

For the relative orientation sensor the value of latest reading["quaternion"] represents the rotation of a device hosting motion sensors in relation to a stationary reference coordinate system. The stationary reference coordinate system may drift due to the bias introduced by the gyroscope sensor, thus, the rotation value provided by the sensor, may drift over time.

That looks like it'd be great for our use case, but would using the complimentary filter snippet from https://github.com/googlevr/cardboard-vr-display/pull/13#issuecomment-356731088 alleviate that drift?

[kenchris]

Most sensor hubs (as used by Orientation Sensors) use complementary filters or something similar as it is fast and easy to implement in hardware, and most likely attempt to remove drift, thought that can never fully be removed. Absolute Orientation Sensors can sometimes sync (ie. using the magnetic north then a quick movement is made).

I wrote an explainer about motion sensors here https://www.w3.org/TR/motion-sensors/

[kenchris]

The Magnetometer is running at low frequency (maybe @owencm remembers why) but it can still be used to measure drift over time (if calibrated properly)

[jsantell]

@kenchris thanks for the info! I'll try a RelativeOrientationSensor patch and see how that performs

[kenchris]

OrientationSensors vs complementary filters with Gyro/Accel

Orientations sensors will use the sensor hub (hardware) if available or do the sensor fusion in code (Probably Java or C++ code on Android) which will presumable be a complementary with bios 98%. Sensor hubs most likely do the same, just in hardware or microcode

[kenchris]

You can play around with https://sensor-compass.appspot.com

The UI is not the best in the world though :-)

[jsantell]

The RelativeOrientationSensor is working pretty well, some issues to resolve with syncing the coordinate system with WebVR's, but looking good :+1:

[pozdnyakov]

some issues to resolve with syncing the coordinate system with WebVR's, but looking good :+1:

@jsantell, we're currently working on solving syncing the coordinate system issue, pls let us know if you have any thoughts on ways to solve it (or maybe you already have a preferred API solution in your mind :) )

[jsantell]

@cvan Thanks for the feedback and testing!

NOTE: I had to proxy this through ngrok to get a secure context (i.e, the examples pages served over HTTPS) because of the Feature Policy restrictions.

I was able to get around this using localhost (testing via adb on Pixel) FWIW.

@alexshalamov There is a fullscreen interstitial displayed when entering VR using the CardboardVRDisplay, prompting the user to rotate the screen, similar to Daydream. Regarding the sensor fusion, working on using the RelativeOrientationSensor as well with this! @pozdnyakov I'll ping if a good pattern surfaces on that transformation :+1:

[kenchris]

I already do some transformation in the polyfill: https://github.com/kenchris/sensor-polyfills/blob/master/src/motion-sensors.js#L310

Basically what I believe makes the most sense to have by default because it makes things behave as most would expect.

[jsantell]

Uploaded a new patch, using RelativeOrientationSensor -- seems more smooth than using fusion on Gyroscope/Accelerometer! The new PoseSensor class here uses the RelativeOrientationSensor, with falling back to the devicemotion FusionPoseSensor class giving us a nice separation of the two implementations. A bit rusty on some of these quaternion transforms, I'm sure there are some better ways here, any ideas? But works in all orientations with solid performance.

Feelin' good about this, ya'll -- thanks for all the feedback and help so far!

[jsantell]

@cvan does AFrame have any explainers on feature policy changes that are incoming that'd affect developers using iframes? Would be a valuable collab (probably all of WebXR in general) to explain why iframe usage will break without using allow=. Could alert a more actionable message here when running into permission issues while being in an iframe that could help point devs to the right place

[jsantell]

Repushed updates; thanks for review @cvan! Only outstanding thing is whether or not the errors from sensor.start() are try/catchable via notify error which:

Fire an event named "error" at sensor_instance using SensorErrorEvent with its error attribute initialized to error.

[jsantell]

@cvan want to give this one last look? I think it's pretty solid, permissive fallback to devicemotion in iframes (due to Chrome m63 only supporting main frames) and the permission logic in general could use another safety check, I think