kasvii / PMCE

[ICCV 2023] PyTorch Implementation of "Co-Evolution of Pose and Mesh for 3D Human Body Estimation from Video"
https://kasvii.github.io/PMCE
MIT License
150 stars 5 forks source link

Pretrained Model Details | Requesting Clarification #9

Open AmoghTiwari opened 9 months ago

AmoghTiwari commented 9 months ago

Hello,

Thank you for your great work, and making the code publicly available.

In the link provided to the pretrained models, there are multiple pretrained models available. While you mention that for demo purposes, we should use the "mesh_vis.pth.tar" model, it is not clear what the other models are.

My understanding is that the other models are specific models for evaluation on different datasets. If my understanding is correct, then could you please clarify whether you perform individual training and testing for each dataset, or whether you train a single model on a mixture of the datasets and then perform all testing with that single model?

kasvii commented 9 months ago

You are correct. The models are trained and evaluated on different datasets to compare with other methods, as shown in Table 1 of the paper. You can reproduce them under the training instruction. For 3dpw, the model was trained on mixed 2D and 3D datasets, including 3dpw, h36m, mpii3d, coco, and mpii, and tested on 3dpw testing set. For mpii3d, we take 3dpw's checkpoint and directly test on mpii3d testing set. For h36m, the model was trained on h36m training set and tested on h36m testing set. 'mesh_vis.pth.tar' is one of 3dpw's checkpoints. Hope this reply helps you.

AmoghTiwari commented 9 months ago

Thank you for clarifying this in detail :). This makes it quite clear. Closing the issue.

AmoghTiwari commented 8 months ago

As a follow up to my previous query, is this the commonly used strategy adopted by previous works as well? My understanding is that previous works like MPS-Net or TCMR use a single pretrained checkpoint (trained on a combination of datasets) for evaluation across all the test datasets.

kasvii commented 8 months ago

MPS-Net and TCMR also use different settings for diverse test datasets, as indicated here. In contrast to MPS-Net and TCMR, PMCE introduces the pose branch, which is influenced by joint set types. Thus, PMCE mainly follows the settings of the pose-based method, Pose2Mesh.

Dragon2938734 commented 6 months ago

hello, the pretrained model you provide is 393M, but i reproduced following the training introduction is 1.1G, is there any problems cause this difference? Thank you and look forward to your reply!

kasvii commented 5 months ago

hello, the pretrained model you provide is 393M, but i reproduced following the training introduction is 1.1G, is there any problems cause this difference? Thank you and look forward to your reply!

Hello, to save memory for easy downloading, the uploaded pretrained model only contains the model parameters. In contrast, the reproduced one contains additional optimizer parameters.