Finding Orientation of 3D Pose

mmbaguette commented 2 years ago

Using the 3D pose estimation model, how can I calculate the yaw, pitch and roll rotation Euler angles?

For example, if I raised my left arm in front of me, level to the ground, how can I determine that the pitch in this example is 90 degrees?

The only issue these threads is address is for the face mesh estimation. I’m looking for the entire pose.

Python 3.9 on Windows.

Edit: I’m not knowledgeable in 3D coordinate physics like matrices. Please tell me more than just giving me a resource where I can calculate these angles.

Thanks.

cgtinker commented 2 years ago

in your case you can use the camera look at concept. let's say they shoulder is the eye and the elbow the target.

def vec_length(v: np.array):
    return np.sqrt(sum(i**2 for i in v))

def normalize(v):
    norm = np.linalg.norm(v)
    if norm == 0: 
       return v
    return v / norm

def look_at(eye: np.array, target: np.array):
    axis_z = normalize((eye - target))
    if vec_length(axis_z) == 0:
        axis_z = np.array((0, -1, 0))

    axis_x = np.cross(np.array((0, 0, 1)), axis_z)
    if vec_length(axis_x) == 0:
        axis_x = np.array((1, 0, 0))

    axis_y = np.cross(axis_z, axis_x)
    rot_matrix = np.matrix([axis_x, axis_y, axis_z]).transpose()
    return rot_matrix

this function returns a 3x3 rotation matrix from which you can calculate a quaternion.

sureshdagooglecom commented 2 years ago

Hi @mmbaguette , Can you provide code changes to investigate further on this issue.

mmbaguette commented 2 years ago

I will take a look soon today, as I'm occupied atm. Wdym by code changes?

google-ml-butler[bot] commented 2 years ago

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you.

mmbaguette commented 2 years ago

Hey @cgtinker, your code seems to be responsive to my movements. However, the numpy array returned gives me three different arrays each containing three values. What does this mean, and how do I convert it to degrees?

What I tried:

lf_sh = pose_landmarks["left_shoulder"]
lf_wr = pose_landmarks["left_wrist"]

orient = look_at(np.array([lf_sh["x"], lf_sh["y"], lf_sh["z"]]), 
    np.array([lf_wr["x"], lf_wr["y"], lf_wr["z"]]))

print(np.rad2deg(orient)) # convert each value from radians to degrees

Example output:

[[ 37.91493195   4.95226    -42.43288736]
 [-42.43288736   4.42497819 -37.91493195]
 [  0.          56.51536679   6.68687933]]

Are the values in each array radians? Thanks.

Edit: I don't know what a quaternion is. Can you help me find the X, Y and Z angles from your function anyway? My limit is 2D vectors.

cgtinker commented 2 years ago

Hey @cgtinker, your code seems to be responsive to my movements. However, the numpy array returned gives me three different arrays each containing three values. What does this mean, and how do I convert it to degrees?

What I tried:
lf_sh = pose_landmarks["left_shoulder"]
lf_wr = pose_landmarks["left_wrist"]

orient = look_at(np.array([lf_sh["x"], lf_sh["y"], lf_sh["z"]]), 
    np.array([lf_wr["x"], lf_wr["y"], lf_wr["z"]]))

print(np.rad2deg(orient)) # convert each value from radians to degrees
Example output:
[[ 37.91493195   4.95226    -42.43288736]
 [-42.43288736   4.42497819 -37.91493195]
 [  0.          56.51536679   6.68687933]]
Are the values in each array radians? Thanks.

Edit: I don't know what a quaternion is. Can you help me find the X, Y and Z angles from your function anyway? My limit is 2D vectors.

https://stackoverflow.com/questions/15022630/how-to-calculate-the-angle-from-rotation-matrix

Don't have code laying around.. Check the link, it's pretty easy to do but U may have to store a combat so U know when it's above or below 180deg.

sureshdagooglecom commented 2 years ago

Hi @mmbaguette . Hope this https://github.com/google/mediapipe/issues/3226#issuecomment-1096846680 helped to resolve this issue .

google-ml-butler[bot] commented 2 years ago

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you.

google-ml-butler[bot] commented 2 years ago

Are you satisfied with the resolution of your issue? Yes No