search

zju3dv / SMAP

[ECCV 2020] SMAP: Single-Shot Multi-Person Absolute 3D Pose Estimation
Apache License 2.0
241 stars 37 forks source link

evaluation about MuPoTs-3d #49

Closed HuiliCui closed 2 years ago

HuiliCui commented 2 years ago

Hi， I want to test the results on the MUPOTS dataset. The script settings are as follows image and in convert.py I use the K from MuPoTs-3D then set eval_mode = 1;is_relative = 0; in mupots-smap.m, The result is very poor! Could you tell me what went wrong? image

raypine commented 2 years ago

test_mode: generate_result do_flip: 1 Then use the code in lib/eval.

HuiliCui commented 2 years ago

image Thank you very much for your reply！Could you provide this file M3E_gt.json？

raypine commented 2 years ago

Basically the file is generated in the same way as the training data files. I upload the file via this link. But I have not checked it totally. You can have a try if possible. P.S. Some keys in the json file are useless.

HuiliCui commented 2 years ago

It helps a lot! I have obtained the right results! Could you tell me what should I do about testing the Panoptic dataset?

raypine commented 2 years ago

The model we provided is only for MuPoTS-3D. Unfortunately, it's difficult to find the model trained on the Panoptic dataset now. But according to our experience, to achieve the performance in our paper (even better) is easy, since the distribution of training data is close to that of testing data to some extent, e.g. the camera viewing angle, the environment. If you want to perform the experiments on the Panoptic dataset, you could finetune our provided model or train from scratch on it.

HuiliCui commented 2 years ago

That means the provided model is not trained by panoptic dataset?

raypine commented 2 years ago

Yes, it's trained with MuCo and tested on MuPoTS-3D.

jihao0309 commented 2 years ago

Hello，i followed the test script above, and the visualization of the results is good, both 2d and 3d. Then i want to transform the format of result from json to matlab, i commented the code that i think it is unnecessary in convert.py. image

The results pose2d.mat and pose3d.mat in matlab format were obtained. Next, i run the code mupots_smal.m, set eval_mode=1 and is_relative=1, i guess in this way, i can get the PCK_abs. But the results are worse than the results in paper.The average PCK_abs is about 68. Is there anything wrong with what I'm doing, please point it out, thanks a lot！！！ image

raypine commented 2 years ago

Why do you comment them ? Since we resize the input image to the resolution of 512 x 832, it's necessary to transform the estimation to the actual scale.

jihao0309 commented 2 years ago

Why do you comment them ? Since we resize the input image to the resolution of 512 x 832, it's necessary to transform the estimation to the actual scale.

Sorry, I made a mistake in another place, I can get the right result now, thanks!