search

microsoft / Azure-Kinect-Sensor-SDK

A cross platform (Linux and Windows) user mode SDK to read data from your Azure Kinect device.
https://Azure.com/Kinect
MIT License
1.49k stars 620 forks source link

ankle orientation seems to actually be a missing "heel" joint #1637

Open diablodale opened 3 years ago

diablodale commented 3 years ago

I have evidence that suggests the K4ABT_JOINT_ANKLE_LEFT and K4ABT_JOINT_ANKLE_RIGHT joints output incorrect rotations. They appear to actually be rotations pointing towards a missing heel joint.

Reproduced using both the k4abt_simple_3d_viewer.exe tool in k4abt sdk, and in my own code, Perhaps when the models were trained on (synthetic) data or later processing, the team used, or dropped, a heel joint.

image

Setup

Repro

  1. Wear tight clothing on waist, legs, and no shoes. This is to maximize the detection of joints.
  2. Position the Kinect sensor ~30cm above the floor.
  3. Position a chair ~150cm in front of the sensor.
  4. Rotate the sensor slightly up and towards you in a sitting position.
  5. Sit in the chair
  6. Run from the k4abt sdk install location tools\k4abt_simple_3d_viewer.exe
  7. Maximize the main window and press b on your keyboard to display axes
  8. Lift your right foot as depicted in the above screenshot.
  9. Slowly pivot your ankle by lifting and lowering your right forefoot
  10. Watch closely the rotation axes on the right ankle and the bone connecting the right ankle -> toes

Result

None of the 3 rotation axes on the right ankle align with any bone. Kinect Azure typically aligns local X axis with bones. Instead, the ankle rotation appears to form a right angle with the foot rotation with both their X axis meeting at a "heel". This missing "heel" then aligns the X (red) axis on this missing ankle->heel bone.

Expected

Ankle rotation to align its X (red) axis on the bone from ankle -> foot. I am not expecting a new joint "heel" to be created.

Notes

This issue is also on the left side. It also reproduces on my own code. In the following picture you can see a rendering of the left foot raised and you can see the bone from foot -> ankle is instead rotated from foot -> missing heal.

image

qm13 commented 2 years ago

In the attached picture, angle (1) will always be 90°, despite you moving your foot to (2)nd or (3)rd direction. Which means that there is "imaginary" joint, where the 90° angle is created. Heel joints