Interpretation of the output and constraining joints with < 3 DoF

[AJDA1992]

First really nice work! Trying to understand how to interpret the outputs here. From the outputs what would be interpreted as trunk flexion/extension vs trunk lateral bending vs trunk torsion? Also, Do you constrain for joints that only have 1 or 2 axis of rotation?

[AJDA1992]

@TemugeB Also correct me if I am wrong please, it seems that the angles that are reported mean how much rotation any axis has away from the fixed t pose correct? Could we easily convert this to where the fixed pose is say an anatomically neutral posture?

[TemugeB]

Hi. Thanks for checking out this repo. However, as I stated in the readme, this repo is mostly intended to re-target a pose onto a standard skeleton. Constraining angles so that it makes anatomical sense is unfortunately a lot harder. But it could probably be done with some extra work. I'm no expert in anatomy so there are no constraints on the joints.

To answer your questions:

1. I don't know what flexion, torsion and lateral bending are.
2. No, joints with 1 or 2 DOF are not constrained. However, look below for suggestion on how to constrain this.
3. The T pose is used in the animation industry and allows for a simple definition of the axes for each joint. This however does not constrain joint angles. Constraining joint angles is not really necessary when working with mocap data.
4. Basic pose can be changed. You just need to modify the offset directions. Note however that the offset directions are with respect to their parent joints' defined axes. This can get confusing because if the parent joint is already rotated, then your child joint offset should be defined in this rotated axes. This is why T-pose is used because we can assume all parent joints are not rotated.

If you want to constrain joints, then I suggest you define the basic pose with care. The joint axes should be defined in such a way that makes sense to you or makes sense anatomically. Once joint axes are defined properly, you can calculate joint angles based on how many DOF it has. 3DOF can use the same formulation that I'm using now. For 1DOF, you can use simple cosine angle between the parent and child joints. 2DOF is something that you would have to figure out by yourself. You may also need to define different rotation orders for different joints. In any case, anatomically constrained joint angles is hard to obtain and if another work doesn't exist already that does this, then you probably will have to write it yourself. You seem to know enough about anatomy so you're probably the best person to do this. I suggest you use this repo as some sort of basic guide and try to build your own.

It should be noted that joint angles are not unique. Meaning different rotation degrees can result in the same final pose. Additionally, rotation order will also change the rotation angles. So this is an additional layer of complexity.

Sorry I'm not really much of a help beyond this. If you need help with the math, I might be able to give you some. Good luck

[TemugeB]

Some additional points are:

1. 3D joint position is usually not obtained correctly. If you use a pose estimation network as I did, then it is estimating a joint inside the body based on the surface of the body. So usually the joint position is not correct. If you use mocap equipment that uses surface markers, then you again get the same problem, where you're estimating a joint inside the body based on surface markers. So you can't fully trust the 3D points for the joints. Problem is compounded if camera calibration is poor and adds uncertainty to the triangulated 3D point.
2. When you constrain a joint angle, the poses might not match when you only rotate along a certain direction. So 1DOF and 2DOF might not result in correct pose. This is especially true if the 3D points for the joints are not certain. In this case, how to deal with constraints become very difficult. The joints are dependent on the pose of their parents, so that if one joint is incorrectly rotated due to angle constraint, the rest of the joints on that chain will also include that error. So even with 1DOF joints, you might need to allows 3DOF angles.

As I said, this constrained pose angles is a hard problem.

[AJDA1992]

@TemugeB Thanks for the thoughtful responses. First, after some thought at least for the skeleton as you defined it, constraining doesnt really make sense because all they joints are ball type and so have 3DOF. So if you can illuminate this point anymore: how should one interpret the angle outputs produced by this script? Is it essentially what amount of rotation is required by fixing some axis to move between the pose and the base skeleton?

[aali26]

Based on my understanding, we are working with the skeleton as a multi-end effector problem, where given the end joint position (foot, head, hands) how much each limb should be rotated from the rest pose to the target pose. In this case of the hand, we are calculating all the joints' rotation for all joints in the chain of that end effector. In other words, we are calculating the inverse kinematic given the joint location.

@TemugeB, since in your implementation you are not calculating the angles of the end effectors. If I want to include the rotation for even the end effectors like (head, feet, and hands) will the cosine angle be enough or not?

[AJDA1992]

@aali26 so you are saying in this case we are not calculating the relative angle between the parent and the child joint but rather whata is the rotation from any given joint to that same joint in the rest pose? Also, I was under the impression that we are calculating the joint for the end effectors like hands and feet?