search

isarandi / metrabs

Estimate absolute 3D human poses from RGB images.
https://arxiv.org/abs/2007.07227
MIT License
468 stars 69 forks source link

No tracking reasoning of hand joint #29

Closed psiydown closed 2 years ago

psiydown commented 2 years ago

First I would like to say great work.The hand or palm joint follows the wrist and has no self motion tracking, Are there any options or modifications that can be turned on? Is there any plan for an accelerated version based on TensorRT? Thank you!

isarandi commented 2 years ago

Hi, this is a known limitation that comes from the training data. Specifically, the SMPL joints are learned from the SURREAL dataset, which is a synthetic dataset that does not articulate the hand joint compared to the wrist. This may be updated in the future using a suitable dataset, but I'm not sure when I'll have time to experiment with that. So (for now at least), these models are not great for hand analysis. I'd recommend checking out other methods that specifically focus on hand pose estimation if you need to analyze hand gestures (it's kind of its own task/niche/subcommunity in the literature, though it can certainly be performed jointly with body pose estimation).

TensorRT: unfortunately I won't have time to do this on the short term, but some other users have experimented with it, check out those Github issues, perhaps you find something useful there.