search

geopavlakos / hamer

HaMeR: Reconstructing Hands in 3D with Transformers
https://geopavlakos.github.io/hamer/
MIT License
350 stars 30 forks source link

About video consistency #65

Closed haonanhe closed 2 weeks ago

haonanhe commented 3 weeks ago

HaMer performs really well on estimation of a single image. However, when implemented on videos, inconsistency of mano poses and betas can be observed between frames. Is it possible to improve the consistency?

geopavlakos commented 3 weeks ago

HaMeR operates on each frame independently, so unfortunately the temporal consistency is not guaranteed. You could consider post-processing the HaMeR results, by optimizing them and encouraging the whole sequence to have the same shape parameters, and/or encourage neighboring frames to have similar poses.