model_hand.json vs. vanity_bones.json

[hinmanj]

What's the difference between these two assets. Is it safe to assume the model_hand.json is the hand model being used in all the projects for fitting to the depth maps, and as a base for the "segment_scale" variable being used everywhere?

[melax]

Good question.

First let me share a picture I specifically made for you in order to help show the bone numbering:

Yes the model_hand.json is the main model file used for the fitting geometry. the "segment_scale" defaults to 17 which is meant to be the size in cm of the geometry model in the model_hand.json file.

The geometry in the data file is low-poly control cages from which 2 iterations of (non-interpolating) Catmull-Clark subdivision are done before resulting in the actual geometry bones used for the tracking.

To answer your question about the (programmer-art) file vanity_bones.json... These were only used once for visualization purposes in order to quickly create some hour-glass-shaped "bone" looking bones for when a user placed his hand in an "x-ray hand scan" stage demo that would strip away the skin and show bone geometry.

The vanity_bones have no algorithmic purpose. Feel free to ignore this file.

[hinmanj]

While on the topic of the model_hand.json, I was hoping you could verify my assumption with the joint information below:

"joints": [ { "jointframe": [0,0,0,1], "p0": [-0.0143744,-0.00155609,0.242809], "p1": [0,0,0], "rangemax": [70,30,0], "rangemin": [-70,-20,0], "rbi0": 0, "rbi1": 1 }, .... ]

In the example joint above, I assume "p0" corresponds to the 3D position in hand model space of the connection point between the top of the wrist and the bottom of the palm rigidbodies? And in that case, is "p1" just there but not being used. If so, what is an exemplar of "p1" being used for anything?

And for the "range" values, what exactly do those dictate? (I'm assuming they're the amount pull the physics simulation will allow, or something to that effect)

[melax]

The joint information describes the attachment between two rigid bodies. Without joints, there is nothing holding the bones together and they would just fall apart. I specifically created an image sequence just for you to illustrate this fact:

Clearly this is not the desired behavior. Therefore we have joints to constrain the motion of the bones.

Within the context of physical simulation it is better to think of this as a binary relationship, instead of a parent-child relationship. Attachment points p0 is local to rigidbody0 and p1 is local to rigidbody1. The geometry in the file happens to have been authored in typical "rig" space where the origin of a bone happens to be located where it is attached to its parent. For example, the origin (0,0,0) of the palm is at the base of the palm. Consequently, p1 will be 0,0,0, and p1 will be where that bone is attached to its parent-bone.

Internally, the rigidbodies will use coordinate frames adjusted by their center-of-mass which simplifies all the code.

[melax]

The joints used here assume attached positions and some range of motion. the range{min,max} are how far the bones can rotate in degrees relative to each other in the x, y, and z axes respectively. The above limited-2DOF example you show has 140 degree range about the x axis, 50 degrees about the y axis, and none about z.

The palm and first bone of each finger will typically be attached with 2dof, and the upper finger bone joints will have 1dof.

There are some more complicated relationships that restrict motion that are a function of more than 2 bone positions. For example the last two joints on each finger typically have the same angle. Such relationships are enforced by adaptively modifying these ranges in the code within the function HandModelEnhancements().