Accelerometer/orientation unforms ignore screen rotation

[AntiBlueQuirk]

It seems that the various sensor outputs for orientation ignore the rotation of the screen, and since there's no way to determine screen rotation in the shader, this means that things like the gravity vector are just wrong when the screen is rotated into a landscape orientation. You can see this by loading up the Gravity Vector preset. When the device is in a portrait configuration, the screen corresponds to the device orientation quite nicely, but when the device switches to landscape configuration, the motion on the screen stops making sense.

I'm not sure exactly what sensor outputs need to change, but I've had a great deal of difficulty getting any of the accelerator based uniforms to output logical values switching between portrait and landscape configuration.

Other than that, awesome program!

[markusfisch]

Yes, that's right, with exception of the orientation uniform, no sensor input is remapped according to screen orientation. Did you try the Orientation sample shader from Menu -> Load sample -> Orientation by the way?

It should be possible to remap the linear, gravity and magnetic vectors with the orientation uniform but that can (and should be) done in the app too.

[AntiBlueQuirk]

Hmm. I was under the understanding that the orientation vector was subject to this problem, too, based on my tests. I've also had a lot of trouble using orientation for my shaders, because it seems really unstable in some orientations, possibly related to gimbal lock.

I just brought this up, because I thought it was more intuitive for gravity and its family to be in shader space instead of device space. I'm also not sure how to use orientation to transform gravity into shader space, since orientation doesn't contain any information about the rotation of the screen, just the device's orientation in "earth-space".

[markusfisch]

The orientation uniform gives "the device's orientation based on the rotation matrix". The rotation matrix is computed "by transforming a vector from the device coordinate system to the world's coordinate system which is defined as a direct orthonormal basis, where:" (pasting from the Android documentation)

• X is defined as the vector product Y.Z (It is tangential to the ground at the device's current location and roughly points East).

• Y is tangential to the ground at the device's current location and points towards the magnetic North Pole.

• Z points towards the sky and is perpendicular to the ground.

The app invokes SensorManager.getRotationMatrix() to compute the rotation matrix from the gravity and geomagnetic vector. That rotation matrix is "the identity matrix when the device is aligned with the world's coordinate system, that is, when the device's X axis points toward East, the Y axis points to the North Pole and the device is facing the sky".

Then, SensorManager.getOrientation() is called to compute the device's orientation based on this rotation matrix. The three members of orientation are (pasted from the Android documentation):

• Azimuth, angle of rotation about the -z axis. This value represents the angle between the device's y axis and the magnetic north pole. When facing north, this angle is 0, when facing south, this angle is π. Likewise, when facing east, this angle is π/2, and when facing west, this angle is -π/2. The range of values is -π to π.

• Pitch, angle of rotation about the x axis. This value represents the angle between a plane parallel to the device's screen and a plane parallel to the ground. Assuming that the bottom edge of the device faces the user and that the screen is face-up, tilting the top edge of the device toward the ground creates a positive pitch angle. The range of values is -π to π.

• Roll, angle of rotation about the y axis. This value represents the angle between a plane perpendicular to the device's screen and a plane perpendicular to the ground. Assuming that the bottom edge of the device faces the user and that the screen is face-up, tilting the left edge of the device toward the ground creates a positive roll angle. The range of values is -π/2 to π/2.

Please note that Roll ranges from -π/2 to π/2 while Azimuth and Pitch range from -π to π.

When I checked the implemenation, I found an error in remapping the rotation matrix according to the rotation of the screen (deviceRotation in the code). Fixed in fc2160c. I just published a new beta version (2.10.4) with those changes so you can try it without that error.

SensorManager.getRotationMatrix() also gives an inclination matrix I which is "transforming the geomagnetic vector into the same coordinate space as gravity (the world's coordinate space). I is a simple rotation around the X axis". From that matrix a geomagnetic inclination angle in radians can be computed and it would be quite easy to expose that too if you think it would be useful.

[AntiBlueQuirk]

I think my issue is that I've been trying to construct a matrix to convert device space to world space, and looking at the way the values of orientation are calculated, I don't think it contains enough information to reliably reconstruct this matrix. Would it be possible to expose the value of rotation matrix SensorManager.getRotationMatrix() returns as a uniform? That would make it very easy to convert from device space to world space.

[markusfisch]

Yes, of course, it's quite easy, actually. I did just that in a branch and published a new beta, version 2.10.6, from that branch so you can see if it works for you. Should become available in the next few minutes.

In version 2.10.6, there are two new uniforms: mat3 rotationMatrix and mat3 inclinationMatrix that are the transposed matrices from SensorManager.getRotationMatrix(). I'm not quite sure if it's really useful to transpose the matrices and if you like them not transposed, please let me know.

Also, to avoid confusion with the vec3 rotation uniform, I've decided to rename that to vec3 gyroscope. So that's a potential breaking change.

I really should add an example shader rendering a 3D object that is transformed by the orientation uniform.

[Asmageddon]

How would I use any of these to project a cubemap skybox based on the device orientation? I've been banging my head against this for a while but it's just above me.