search

zc-alexfan / hold

[CVPR 2024✨Highlight] Official repository for HOLD, the first method that jointly reconstructs articulated hands and objects from monocular videos without assuming a pre-scanned object template and 3D hand-object training data.
https://zc-alexfan.github.io/hold
MIT License
271 stars 7 forks source link

Large dynamic interaction examples #8

Closed anilesec closed 1 month ago

anilesec commented 2 months ago

Hi @zc-alexfan

In qualitative results, I could not find large hand-object dynamic interaction examples(eg. for sequence ShSu10..14). Have you tried on this kind of sequence? I am curios :)

zc-alexfan commented 2 months ago

hi @anilesec, for larger objects, usually, it will work quite well because there will be less occlusion. However, I don't think there are large objects in HO3D.

anilesec commented 2 months ago

@zc-alexfan thanks for the response. I think I didn’t frame my question well. Sorry! I mean what happens if the hand articulation is high such that the fingers gets occluded by objects.

Also, if the hand is largely occluded by object then off the shelf hand pose estimator is likely to struggle. I wonder what happens in this case.

thanks in advance!

zc-alexfan commented 2 months ago

If for a pixel the object occludes the hand, assuming the object is in front of the hand for that pixel in our model, then the gradient will be updated for the object, not the hand so it will only affect the object.