Reproduction of the numbers in the paper

[Seethevoice]

Thanks for sharing your great work.

When I train TCMR with the code (default training configs) and datasets provided here, I get

{'mpjpe': 98.79354, 'mpjpe_pa': 56.348724, 'accel_err': 7.208395533521851, 'mpvpe': 119.89002}

, which is worse than what is provided in Table 4 (3DPW test set).

Can you please provide suggestions or any additional clarifications? Thank you.

[hongsukchoi]

Hi,

to reproduce the result, you should use the exact config file. In your case, it will be repre_table4_3dpw_model.yaml.

[Seethevoice]

Hi,

Yes, the config file "repre_table4_3dpw_model.yaml" is what we use. However, the result we got is {'mpjpe': 98.79354, 'mpjpe_pa': 56.348724, 'accel_err': 7.208395533521851, 'mpvpe': 119.89002}, which is worse than what is provided in Table 4 (3DPW test set).

[hongsukchoi]

Hi @Seethevoice,

I just re-trained TCMR with repre_table4_3dpw_model.yaml, and got {'mpjpe': 86.45539, 'mpjpe_pa': 52.66955, 'accel_err': 6.771879402323846, 'mpvpe': 103.22485}. This is the same with the performance reported in the current arxiv version.

I'm not sure why you are getting different results. Could you check these two things?

1. Again, you should train with the correct config file. There is a hard coding related to the config file's name. Thus you should follow the suggested command python train.py --cfg ./configs/repr_table4_3dpw_model.yaml --gpu 0, rather than changing the default config file's content.

2. The numbers shown in the end of the training is outcome on the 3DPW validation set, not the test set. After training, the checkpoints are saved in ${ROOT}/experiments/{date_of_training}/. Change the config file's TRAIN.PRETRAINED with the checkpoint path and follow the evaluation command.

[Seethevoice]

Hi,

I still have the same problem. I follow the configuration file "repre_table4_3dpw_model.yaml" to train the model. However, the reproduced results are still different from Table 4 (3dpw test set). I found that the author used different training sets to train the model for a specific test set. Therefore, I want to confirm with the author whether the following dataset is used to train the model for the 3dpw test set.

insta_train_db.h5 posetrack_train_occ_db.h5 mpii3d_train_scale12_new_occ_db.pt h36m_train_25fps_occ_nosmpl_db.pt 3dpw_val_db.pt 3dpw_test_db.pt

Thank you very much.

[hongsukchoi]

Hi @Seethevoice, thanks for your interest.

Yes, we used different training sets to train the model for a specific test set.

We used

insta_train_db.h5 3dpw_train_occ_db.pt mpii3d_train_scale12_occ_db.pt h36m_train_25fps_occ_nosmpl_db.pt

for the 3dpw test.

Of course, we did not use the validation or test sets of 3dpw in training.

[hongsukchoi]

If you want, I can check out your codes. Looking at the datasets you listed, I think you have been using different train data. If you follow the protocol (just run python train.py --cfg ./configs/repr_table4_3dpw_model.yaml --gpu 0), you will use the exact train datasets I used.

[Seethevoice]

Thank you for your reply, I have solved the problem.

[xiangcece]

Hello How did you solve the problem？ I can't reproduce the effect so far. Thank you

谢谢你的回复，我已经解决了这个问题。

[xiangcece]

Hi @Seethevoice, thanks for your interest.

Yes, we used different training sets to train the model for a specific test set.

We used

insta_train_db.h5 3dpw_train_occ_db.pt mpii3d_train_scale12_occ_db.pt h36m_train_25fps_occ_nosmpl_db.pt

for the 3dpw test.

Of course, we did not use the validation or test sets of 3dpw in training.

Hello, can you send the train_log.txt file?

[hongsukchoi]

I don't have it now. Will you explain or provide your situation?

[xiangcece]

I used the correct data set, but I still can't achieve the effect in the paper. I don't know what's wrong. So I want to have a look at your log file.

[hongsukchoi]

Which table are you trying to reproduce? Did you use NeuralAnnot?

[xiangcece]

table 4 . I want to realize the effect of 3dpw data set.

[xiangcece]

您要复制哪个表？你用过NeuralAnnot吗？ Hi, can you help me to see where I am going wrong, I changed the value of NUM_WORKERS because of the device.Thank you very much！ python train.py --cfg ./configs/repr_table4_3dpw_model.yaml --gpu 0 python evaluate.py --dataset 3dpw --cfg ./configs/repr_table4_3dpw_model.yaml --gpu 0 train_log.txt

{'mpjpe': 87.92476, 'mpjpe_pa': 54.182583, 'accel_err': 6.926254890195235, 'mpvpe': 104.81117}

[hongsukchoi]

您要复制哪个表？你用过NeuralAnnot吗？

I don't speak chinese. Anyway you don't have to use NeuralAnnot to reproduce Table 4.

Your commands look good. I found the train_log.txt that I used to answer @Seethevoice. train_log.txt

You are training longer (100epochs) than the original cfg file (30epochs), but it seems that's not the problem. If you are sure that you have not changed any other codes, the only difference seems to be the GPU. Do you have 2080Ti?

[xiangcece]

Thank you very much for replying to my question, I already got the effect of the paper, I changed the value of NUM_WORKERS because of insufficient shared memory. I found that the effect of the paper can only be obtained when NUM_WORKERS =16, and the smaller the value of NUM_WORKERS , the greater the deviation. Thanks again for your reply！！！

[hongsukchoi]

I see. maybe the random sampling was the problem

[Seethevoice]

I see. maybe the random sampling was the problem

It's a good work anyway, who wouldn't like it? ;)

[xljh0520]

Hi, thanks for your work! I didn't find the h36m_train_25fps_occ_nosmpl_db.pt in your google drive. Is there a missing file?

[hongsukchoi]

@xljh0520

Yeah, I updated the code. SMPL parameters were not available, but since they are available, I updated the data and codes.

[xljh0520]

@xljh0520

Yeah, I updated the code. SMPL parameters were not available, but since they are available, I updated the data and codes.

Thanks for your reply. Since you updated the code, so now what do I need to download to reproduce your results in table 4 and 6?

[hongsukchoi]

the files without ‘nosmpl’