alekhra
commented
2 years ago Hello, I am sorry for creating this issue. I am able to replicate your results on both 3dpw and Human3.6. The issue was because of the PyTorch version. I had Cuda 10.2 and I used python to create an environment, building PyTorch from source using pip. But using pip I was getting the distribution for torch==1.11.0+cu102. This torch did not have torch.cuda.comm.broadcast method. I did not handle this issue properly and hence the errors were coming too high. I fixed it using PyTorch 1.6.0. Below are the results with 1 epoch:
Testing on 3DPW
100%|████████████████████████████████████████████████████████████████████████████████████████████████████| 35515/35515 [54:39<00:00, 10.83it/s] Evaluation start... XYZ_14 PA-MPJPE: 48.995369 | MPJPE: 79.220801, x: 29.987403, y: 32.76, z: 53.334388
Test result is saved at exp/test_3d_kpt.json Evaluation start... XYZ_24 PA-MPJPE: 53.598101 | MPJPE: 77.350245, x: 28.186361, y: 32.94, z: 52.341842
Test result is saved at exp/test_3d_kpt.json PVE: 93.4598140219232
gt 3dpw err: 48.99536917892333
Testing on Human3.6M
100%|████████████████████████████████████████████████████████████████████████████████████████████████████| 25844/25844 [37:00<00:00, 11.64it/s] Evaluation start... XYZ_14 Protocol 2 error (MPJPE) >> PA-MPJPE: 34.027019 | MPJPE: 52.986037, x: 16.649398, y: 26.24, z: 34.408102 Directions: 52.23 Discussion: 51.00 Eating: 48.30 Greeting: 52.03 Phoning: 56.44 Posing: 48.39 Purchases: 51.84 Sitting: 61.64 SittingDown: 64.38 Smoking: 52.00 Photo: 55.95 Waiting: 49.29 Walking: 45.24 WalkDog: 56.74 WalkTogether: 49.50 Test result is saved at exp/test_3d_kpt.json Evaluation start... XYZ_24 Protocol 2 error (MPJPE) >> PA-MPJPE: 34.839931 | MPJPE: 50.760115, x: 14.669206, y: 25.10, z: 33.688910 Directions: 51.25 Discussion: 49.74 Eating: 47.03 Greeting: 49.42 Phoning: 54.12 Posing: 46.59 Purchases: 46.44 Sitting: 60.07 SittingDown: 60.00 Smoking: 50.74 Photo: 53.32 Waiting: 46.60 Walking: 42.44 WalkDog: 52.56 WalkTogether: 47.32 Test result is saved at exp/test_3d_kpt.json
gt h36m err: 34.02701862861743
This was the problem from my side and I want to thank you for your patience and time. Closing the issue.
Hello, I am using validation script to produce the results as reported. For running validation script I am using : ./scripts/validate_smpl.sh ./configs/256x192_adam_lr1e-3-res34_smpl_24_3d_base_2x_mix.yaml ./pretrained_res34.pth , with the same configurations. I have downloaded the pretrained model from https://drive.google.com/file/d/1SoVJ3dniVpBi2NkYfa2S8XEv0TGIK26l/view (as provided). Even for running for 200 epochs I am getting the results below : ####################################### H36m ####################################### Evaluation start... XYZ_14 Protocol 2 error (MPJPE) >> PA-MPJPE: 441.583574 | MPJPE: 493.770147, x: 167.688077, y: 359.20, z: 182.711807 Directions: 511.48 Discussion: 490.77 Eating: 502.78 Greeting: 523.98 Phoning: 482.57 Posing: 503.79 Purchases: 504.71 Sitting: 486.85 SittingDown: 473.40 Smoking: 474.36 Photo: 533.96 Waiting: 487.39 Walking: 464.98 WalkDog: 497.08 WalkTogether: 501.04 Test result is saved at exp/test_3d_kpt.json Evaluation start... XYZ_24 Protocol 2 error (MPJPE) >> PA-MPJPE: 434.536866 | MPJPE: 462.457456, x: 140.025826, y: 357.18, z: 163.964096 Directions: 489.34 Discussion: 464.32 Eating: 466.34 Greeting: 497.36 Phoning: 450.01 Posing: 478.62 Purchases: 474.01 Sitting: 442.66 SittingDown: 426.29 Smoking: 440.27 Photo: 505.36 Waiting: 458.18 Walking: 440.93 WalkDog: 464.75 WalkTogether: 474.51 Test result is saved at exp/test_3d_kpt.json
gt h36m err: 441.58357386250947
####################################### 3dpw ####################################### Evaluation start... XYZ_14 PA-MPJPE: 438.903729 | MPJPE: 462.740392, x: 149.879205, y: 348.55, z: 158.646349
Test result is saved at exp/test_3d_kpt.json Evaluation start... XYZ_24 PA-MPJPE: 426.160066 | MPJPE: 438.464583, x: 124.601933, y: 350.93, z: 136.161026
Test result is saved at exp/test_3d_kpt.json PVE: 473.74147956098835
gt 3dpw err: 438.90372899916605
Can you please help me getting the better results using your pretrained model. Thanks in advance.
p.s. All the scripts are same as provided by you.