search

hongsukchoi / 3DCrowdNet_RELEASE

Official Pytorch implementation of "Learning to Estimate Robust 3D Human Mesh from In-the-Wild Crowded Scenes", CVPR 2022
MIT License
155 stars 15 forks source link

experimental results #7

Closed with-twilight closed 2 years ago

with-twilight commented 2 years ago

Thank you for sharing. I tried to reproduce your experimental results, but both results were far from those given in your paper.

First result: MPJPE from mesh: 493.55 mm PA MPJPE from mesh: 112.53 mm MPVPE from mesh: 529.30 mm

Second result: MPJPE from mesh: 644.32 mm PA MPJPE from mesh: 103.72 mm MPVPE from mesh: 661.93 mm

Can you share the hyperparameter setting or give some instructions? pytorch:1.6,cuda:10.2

hongsukchoi commented 2 years ago

Hi!

Did you see the instructions for reproducing? The result is obviously not related to the hyperparameter. There will be a critical bug.

with-twilight commented 2 years ago

Hi,

May I ask which one might be the document you are talking about?

hongsukchoi commented 2 years ago

It's indicated at the bottom of README. Seems many people are missing this. Maybe I should change the 'Running' to 'Reproducing'?

스크린샷 2022-06-01 22 09 15
with-twilight commented 2 years ago

I noticed this file, the above results are the results obtained after testing the model trained according to the file. I forgot to say that the results in Table 5 are not much different.

hongsukchoi commented 2 years ago

I don't understand... all hyperparamers are as in yaml files or the config file.

Which tables are you trying to reproduce?

with-twilight commented 2 years ago

I would like to reproduce the results of Table 5 and Table 8. The results of Table 5: MPJPE from mesh: 86.37 mm PA MPJPE from mesh: 55.57 mm MPVPE from mesh: 108.73 mm

The results of Table 8: First result: MPJPE from mesh: 493.55 mm PA MPJPE from mesh: 112.53 mm MPVPE from mesh: 529.30 mm

Second result: MPJPE from mesh: 644.32 mm PA MPJPE from mesh: 103.72 mm MPVPE from mesh: 661.93 mm

If the hyperparameters are all given in the config file, is my problem maybe because of the random seed or something else？

hongsukchoi commented 2 years ago

errors like that high only occur if there is a critical bug. I think you must have changed code somewhere. Can you send me your code? redarknight@snu.ac.kr

with-twilight commented 2 years ago

The E-mail was sent to you. Thank you very much.

hongsukchoi commented 2 years ago

there is no 'data' folder in the zip you sent me. I can't run your code.

Anyway I am sure that the error is absurd. It's like testing with un-trained model.