search

SeanChenxy / HandMesh

MIT License
339 stars 69 forks source link

Reproducing results on HO3Dv2 #86

Closed jinhong-ni closed 1 year ago

jinhong-ni commented 1 year ago

Thanks for your work.

I'm trying to reproduce your result on HO3Dv2 as provided in your paper, which has 9.2 mm PA-MPJPE. As there is no code provided for implementation of HO3D dataset, I followed https://github.com/namepllet/HandOccNet/blob/master/data/HO3D/HO3D.py for HO3D and use base_transform function to transform the image as you have done for FreiHAND. I've also removed the mano_to_mpii function when computing hand joints from vertices, as HO3D requires MANO ordering for the resulting joints. Below are the results I received from the official test server:

Evaluation 3D KP results: auc=0.000, mean_kp3d_avg=85.38 cm Evaluation 3D KP PROCRUSTES ALIGNED results: auc=0.561, mean_kp3d_avg=2.52 cm Evaluation 3D KP SCALE-TRANSLATION ALIGNED results: auc=0.067, mean_kp3d_avg=15.10 cm

Evaluation 3D MESH results: auc=0.000, mean_kp3d_avg=85.86 cm Evaluation 3D MESH ALIGNED results: auc=0.759, mean_kp3d_avg=1.21 cm ,

which is significantly worse than the results provided in your paper. Could you please help me finding what might go wrong here? It would also be beneficial if you could release your implementation for evaluation on HO3Dv2, although I totally understand that this might not be convenient. Thanks in advance for your response and assistance!

SeanChenxy commented 1 year ago

Hello, I apologize for the delayed response. I also regret that I am unable to identify any issues with the information provided above.

Shirley-olivia commented 5 months ago

@jinhong-ni Sorry to bother you, I'm also trying to reproduce the result on HO3Dv2, do you follow the augmentation(augmentation&augmentation_2) used in Mobrecon datasets?

jinhong-ni commented 5 months ago

@jinhong-ni Sorry to bother you, I'm also trying to reproduce the result on HO3Dv2, do you follow the augmentation(augmentation&augmentation_2) used in Mobrecon datasets?

It's been a while since I worked on this project, and apology that I could not recall too much detail. I should have been using the augmentation provided in the default config from what I could remember. Hope this can help a bit.

Shirley-olivia commented 5 months ago

@jinhong-ni Sorry to bother you, I'm also trying to reproduce the result on HO3Dv2, do you follow the augmentation(augmentation&augmentation_2) used in Mobrecon datasets?

It's been a while since I worked on this project, and apology that I could not recall too much detail. I should have been using the augmentation provided in the default config from what I could remember. Hope this can help a bit.

Thank you so much!