search

geopavlakos / hamer

HaMeR: Reconstructing Hands in 3D with Transformers
https://geopavlakos.github.io/hamer/
MIT License
326 stars 28 forks source link

Pretrained ViTPose Model needed for training? #44

Closed VimalMollyn closed 3 months ago

VimalMollyn commented 3 months ago

Great work!

Does the ViT-H backbone of hamer need to be initialized with the pre-trained weights of ViTPose? I wasn't able to find this detail in the original paper, so I was curious if this is essential to the model converging.

geopavlakos commented 3 months ago

The model will probably converge even if you don't use a pretrained ViTPose backbone, but it's important to use the pre-trained ViTPose weights to achieve better performance.