Does the ViT-H backbone of hamer need to be initialized with the pre-trained weights of ViTPose? I wasn't able to find this detail in the original paper, so I was curious if this is essential to the model converging.
The model will probably converge even if you don't use a pretrained ViTPose backbone, but it's important to use the pre-trained ViTPose weights to achieve better performance.
Great work!
Does the ViT-H backbone of hamer need to be initialized with the pre-trained weights of ViTPose? I wasn't able to find this detail in the original paper, so I was curious if this is essential to the model converging.