search

geopavlakos / hamer

HaMeR: Reconstructing Hands in 3D with Transformers
https://geopavlakos.github.io/hamer/
MIT License
326 stars 28 forks source link

Present Checkpoint does not match Metrics reported in Table-1 in Paper: arXiv:2312.05251 #37

Closed aviralchharia closed 3 months ago

aviralchharia commented 4 months ago

Hi, I used the standard FreiHAND .pred and .eval code for testing the released model weights, but seems that the released variant is a previous checkpoint of the SOTA (or perhaps I may be using the .eval incorrectly?). Can the authors suggest if any correction is needed in the original FreiHAND .eval code to match the metrics reported in the paper? I directly loaded the HaMeR model to the FreiHAND .pred file, (following the Demo code released by the authors), to make predictions and then ran .eval.

geopavlakos commented 3 months ago

The evaluation code and the checkpoint should be the same. Is it possible that you don't provide the images correctly to the network (e.g., the scale is wrong)? Could you visualize the image that you give as input to the HaMeR network?

geopavlakos commented 3 months ago

We have also updated the repo with the evaluation code.

aviralchharia commented 3 months ago

Thank you so much!