search

jfzhang95 / PoseAug

[CVPR 2021] PoseAug: A Differentiable Pose Augmentation Framework for 3D Human Pose Estimation, (Oral, Best Paper Award Finalist)
MIT License
368 stars 57 forks source link

Can 3D poses with 16 joints be converted back to the 3D poses with 24 joints? #33

Closed AndyVerne closed 2 years ago

AndyVerne commented 2 years ago

Hi, thanks for the great work!

Since the project convert 24 joints 3DHP data to 16 joints accordingly. I'd like to ask that is it possible to convert pose with 16 joints back to 24 joints?

If it's possible, could you tell me the general process?

Best Regards, Look for your reply.

Garfield-kh commented 2 years ago

Hi, thank you for the interest!

  1. If the accuracy requirement of the extra keypoint is not high, maybe you can do interpolation: 3 points (neck, spine, hip) backbone to 5 points (neck, thorax, spine, spine0, hip) backbone by taking the middle point of each segment.
  2. If the accuracy requirement is high, maybe you need to change the 3D pose setting from 16 joints to 24 joints that exactly match with the 2D pose detector in this framework so that the projection augmentation can be applied.
  3. Another approach is try conditional generation (e.g. CGAN) which input 16 joints, and output 24 joints. But I am not sure if this works.

Hope this helps. Thank you~

AndyVerne commented 2 years ago

I've got it! It's so nice of you to give out the advice. And again, thank you so much for the extremely helpful reply.