search

microsoft / Azure-Kinect-Sensor-SDK

A cross platform (Linux and Windows) user mode SDK to read data from your Azure Kinect device.
https://Azure.com/Kinect
MIT License
1.49k stars 619 forks source link

fitting the pose of a body model to the result the poses of the kinect joints #1406

Closed kfarivar closed 3 years ago

kfarivar commented 3 years ago

Describe the question I want to set the pose of a skeleton model (similar to the one Azure kinect uses) to the pose of the body joints given by the sdk. (I have read the Joint documentation (https://docs.microsoft.com/en-us/azure/kinect-dk/body-joints) but I'm still confused.)

my question are:

  1. What does it mean that the joint orientations are relative to the depth sensor coordinate ? does it mean that if I apply the rotation to the depth coordinate system I get the joint coordinate system ? what does a joint coordinate system tell me about the joint ?

  2. how should I apply the quaternion rotations ?

  1. In what order should I apply the rotations to my skeleton ? should I start from parent to child in the kinect's joints table?

Screenshots kinect_smpl

Desktop (please complete the following information):

Additional context The body model I am interested in is the SMPL model but I only use the pose parameters. Although I don't think this would be relevant to the question. I know the joints are not the same (and they have different mechanics) and I will eventually have to interpolate, But even if the joints where the same that still wouldn't answer the questions above.

knat commented 3 years ago

https://github.com/microsoft/Azure-Kinect-Samples/tree/master/body-tracking-samples/sample_unity_bodytracking doesn't answer your questions, maybe it can give you some hints. 3D quaternion is hard for most of us :)

XLY43 commented 2 years ago

@kfarivar Have you found out the transformation between kinect and smpl's axis-angle representation? I'm facing the same problem and couldn't find the answers from documentations.