Obtaining Hand Pitch, Yaw and Roll using JavaScript

[theo-armour]

Using the Leap Motion API for Java you can obtain pitch, yaw and roll of a hand using the following calls:

// hand is a Leap::Hand object
float pitch = hand.direction().pitch();
float yaw = hand.direction().yaw();
float roll = hand.palmNormal().roll();

What is the best (or even just a good) method for obtaining the same values in JavaScript?

[logotype]

I remember it worked the same before, but maybe this functionality isn't implemented yet since the change to gl-matrix?

[logotype]

Pitch: Math.atan2( y, -z );

Yaw: Math.atan2( x, -z );

Roll: Math.atan2( x, -y );

[theo-armour]

Hi Victor

Thank you for the speedy reply.

This question is relates to obtaining the pitch, yaw and roll of a Leap Motion hand object in a designated frame as in (which does not work in JavaScript):

var hand = frame.hands[0];
var pitch = hand.direction.pitch;
var yaw = hand.direction.yaw;
var roll = hand.palmNormal.roll;

Your response - which is accurate - refers to general algebraic calculations and does not relate to particular Leap Motion data.

Pitch, yaw and roll are sailing/aviation terms. In mechanical engineering or computer science we might just say 'rotation angles' and supply a 3D vector.

So, for example, in Three.js terms I am looking for a good way of obtaining rotX, rotY and rotZ for use in the following:

cube.rotation.set( rotX, rotY, rotZ );

Am I being clear?

Theo

PS Did you know that I was the founder of the HK Macintosh User Group?

[theo-armour]

Here's another way of looking at the palm rotation angles issue:

http://jsfiddle.net/theo/6TJNx/

Looking at the demo and wiggling your hands you will see that neither frame.hands[index].direction nor frame.hands[index].palmNormal supply us with the correct rotation angles for the palm.

[The correct rotation angles may well be embedded in frame._rotation which is an array of nine reals with values ranging from 1 to -1. I assume that these may well be three normalized vectors but why does it take three vectors to define the single vector of the three rotation angles is beyond me. On the other hand maybe frame._rotation may have to do something with this frame thing that remains much of a mystery to me.]

Currently, I am using an awful hack which has gimbal lock issues. And I remain very jealous of the other APIs with their simple roll, pitch and yaw call. So any discussion would be appreciated. Who knows? Perhaps somebody might be working on this...

[neptunius]

Hi Theo, I hope this covers most of your questions and issues. :-)

Regarding the absence of pitch, roll, and yaw functions: @joshbuddy removed the Vector and Matrix classes in commit 25d803b to introduce gl-matrix, a high-performance math library, so we don't have to maintain our own. As @theo-armour noticed, it lacks some convenience functions in our homebrew math library that @deckar01 ported from LeapMath.h (C++) to JavaScript. If we want to bring those back, we can revive Jared's implementations and convert them to be compatible with vec3 and mat3 types provided by gl-matrix. Josh and I think it would be best to do this in a separate library to keep the LeapJS client library lean. However, there are likely other vector math utility libraries out there already, maybe one built on gl-matrix, that we can use instead of doing it all ourselves. Let's use existing libraries whenever possible!

Regarding the "gimbal lock" issue with hand rotation: Neither hand.direction nor hand.palmNormal provide enough information to determine all 3 pitch, roll, and yaw values. You'll notice in the Java snippet you copied from our Sample.java code in your first comment, we use hand.direction to obtain the pitch and yaw, but hand.palmNormal to obtain roll. This is because the user's hand is generally pointing toward their screen, so the direction vector is along the roll axis and a roll value obtained from it would be unstable. Similarly, a yaw value obtained from the palmNormal would be unstable. You can get a pitch value from palmNormal, but it will generally be pointing downward, rather than forward, as you likely want. I noticed in your JSFiddle example that you show the yaw from palmNormal is "ok", but this is not what I observe. I also see some flipping issues when the hand's palmNormal is near the neutral position along the negative y-axis.

Regarding the frame rotation matrix: The frame._rotation matrix is a cumulative value that shouldn't be used directly (hence the underscore), it needs to be compared to another frame's rotation matrix to get the change in rotation between them. Read the implementation of Frame.rotation(sinceFrame) for the details. The frame._translation and ._scale contain cumulative values as well.

Alan Davis // Leap Motion SDK Team

[theo-armour]

Hi Alan

Thank you for the very detailed and very understandable explanation. I appreciate the effort

Josh and I think it would be best to do this in a separate library to keep the LeapJS client library lean. However, there are likely other vector math utility libraries out there already, maybe one built on gl-matrix,

Brandon Jones has certainly done a wonderful job with gl-matrix but, yes, more is needed. I will respond to your pondering in a separate issue.

I hope this covers most of your questions and issues

Yes, your reply does cover all the issues in a considered and detailed manner. The thing is that your reply does not answer the question:

What is the best (or even just a good) method for obtaining the same values

[pitch, roll, yaw] in JavaScript?

I have coded some methods that suck. It is not obvious to me as to how to go from the two vectors to the three reals. Given that the Leap Motion team has a deep understanding of the underlying data and code, it would seem to me that you would be the best people to come up with a temporary work around until pitch, roll and yaw can be built into Leap.js.

Theo

On Thu, Aug 8, 2013 at 5:26 PM, Alan Davis notifications@github.com wrote:

Hi Theo, I hope this covers most of your questions and issues. :-)

Regarding the absence of pitch, roll, and yaw functions: @joshbuddy https://github.com/joshbuddy removed the Vector and Matrix classes in commit 25d803bhttps://github.com/leapmotion/leapjs/commit/25d803bto introduce gl-matrix, a high-performance math library, so we don't have to maintain our own. As @theo-armour https://github.com/theo-armournoticed, it lacks some convenience functions in our homebrew math library that @deckar01 https://github.com/deckar01 ported from LeapMath.h (C++) to JavaScript. If we want to bring those back, we can revive Jared's implementations and convert them to be compatible with vec3 and mat3 types provided by gl-matrix. Josh and I think it would be best to do this in a separate library to keep the LeapJS client library lean. However, there are likely other vector math utility libraries out there already, maybe one built on gl-matrix, that we can use inste ad of doing it all ourselves. Let's use existing libraries whenever possible!

Regarding the "gimbal lock" issue with hand rotation: Neither hand.direction nor hand.palmNormal provide enough information to determine all 3 pitch, roll, and yaw values. You'll notice in the Java snippet you copied from our Sample.java code in your first comment, we use hand.direction to obtain the pitch and yaw, but hand.palmNormal to obtain roll. This is because the user's hand is generally pointing toward their screen, so the direction vector is along the roll axis and a roll value obtained from it would be unstable. Similarly, a yaw value obtained from the palmNormal would be unstable. You can get a pitch value from palmNormal, but it will generally be pointing downward, rather than forward, as you likely want. I noticed in your JSFiddle example that you show the yaw from palmNormal is "ok", but this is not what I observe. I also see some flipping issues when the hand's palmNormal is near the "neutral position along the negative y-axis.

Regarding the frame rotation matrix: The frame._rotation matrix is a cumulative value that shouldn't be used directly (hence the underscore), it needs to be compared to another frame's rotation matrix to get the change in rotation between them. Read the implementation of Frame.rotation(sinceFrame) for the details. The frame._translation and ._scale contain cumulative values as well.

Alan Davis // Leap Motion SDK Team

— Reply to this email directly or view it on GitHubhttps://github.com/leapmotion/leapjs/issues/91#issuecomment-22368202 .

[theo-armour]

Hi Alan

Have you had any thoughts about how to answer the question?

All the other APIs give you pitch, roll and yaw, so why can't the LM JavaScript API?

Anyway, I had a look at the issue over the weekend.

Here's what I have so far:

http://jaanga.github.io/gestification/cookbook/pitch-roll-yaw/r1/pitch-roll-yaw.html

The left palm is what I am using when I have Three.js. The right palm is for when I have use plain JavaScript.

The left palm is not perfect and the right palm is even more not perfect.

The essential code I am using for the plain JavaScript is this:

            var pitchAng = Math.atan2( hand.direction[1] , -hand.direction[2] );
            palm2.rotation.x = pitchAng;

            var rollAng = Math.atan2( hand.palmNormal[0], hand.palmNormal[1] );
            palm2.rotation.z = rollAng;             

            var yawAng = Math.atan2( hand.direction[0], -hand.direction[1] );
            palm2.rotation.y = yawAng;

I assume I am beginning to go in the right direction, but I feel sure that the quality of the motion could be significantly improved. Do you have any thoughts as to how to proceed?

Theo

[pehrlich]

I believe this has been resolved with the methods hand.pitch(), hand.roll(), and hand.yaw()?

[theo-armour]

Peter:

I believe this has been resolved with the methods hand.pitch(), hand.roll(), and hand.yaw()?

'Believing' is good.

It's a pitch, roll and yaw in a good direction.

'Resolving' is a byte trickier.

Show me the test case.

Theo

On Tue, May 6, 2014 at 10:06 PM, Peter Ehrlich notifications@github.comwrote:

I believe this has been resolved with the methods hand.pitch(), hand.roll(), and hand.yaw()?

— Reply to this email directly or view it on GitHubhttps://github.com/leapmotion/leapjs/issues/91#issuecomment-42389936 .