About simplebaseline3D

[CHENYUHANG-smp]

Hi, I'm using the 3D lifter simplebaseline3D. When I use the 2D GT as input, I get the good results on h36m validation.

But when I use the HRNet-W32_h36m_256x256 as 2D detector, the MPJPE on h36m validation is quite high, it's quite different from the results in the papers. Why this happened?

[CHENYUHANG-smp]

The results in paper "3D human pose estimation in video with temporal convolutions and semi-supervised training"

[ly015]

Could you please share more details on how you evaluate the model with a 2D detector?

[CHENYUHANG-smp]

Could you please share more details on how you evaluate the model with a 2D detector?

Many thanks for your quick reply.

First, I evaluate the HRNetW32 model on h36m dataset, and get the results file(.json) with PCK 0.98, the command is as follows: python tools/test.py configs/body/2d_kpt_sview_rgb_img/topdown_heatmap/h36m/hrnet_w32_h36m_256x256.py hrnet_w32_h36m_256x256.pth --out results/keypoints_2d/hrnet_h36m.json --eval PCK

Then, I modified the simplebaseline3d_h36m.py configs as follows:

And I found that the results file can not be loaded properly, I just modified datasets/datasets/body3d/body3d_h36m_dataset.py like this:

` def _load_joint_2d_detection(self, det_file): """"Load 2D joint detection results from file."""

det_file is json

    print(det_file)
    joints_2d_list = []
    with open(det_file, 'r') as f:
        data_read = json.load(f)
        batch_num = len(data_read)
        for i in range(batch_num):
            BS=len(data_read[i]['preds'])
            for j in range(BS):
                joints_2d_list.append(data_read[i]['preds'][j])
        # data_read = data_read[0]
        # print(data_read.keys())
        # print(len(joints_2d_list))
        joints_2d = np.asarray(joints_2d_list).astype(np.float32)

    # joints_2d = np.load(det_file).astype(np.float32)
    joints_2d = joints_2d.astype(np.float32)

    return joints_2d

`

Then, evaluate the 3d model on h36m dataset, use the commend: python tools/test.py configs/body/3d_kpt_sview_rgb_img/pose_lift/h36m/simplebaseline3d_h36m.py simple3Dbaseline_h36m_mmpose.pth --out results/keypoints_3d/hrnetw32_simplebaseline_h36m_mmpose.json --eval mpjpe

[CHENYUHANG-smp]

The change of datasets/datasets/body3d/body3d_h36m_dataset.py is like this:

[ly015]

Thanks for the feedback. We are going to need some time to look into this issue. Meanwhile, could you please try if you can get proper evaluation results using the CPN 2D detection results provided by VideoPose3D? The data file could be downloaded from here

[CHENYUHANG-smp]

Thanks for the feedback. We are going to need some time to look into this issue. Meanwhile, could you please try if you can get proper evaluation results using the CPN 2D detection results provided by VideoPose3D? The data file could be downloaded from here

I try the CPN 2D detection results and get the correct results, the MPJPE is much lower:

But I still feel confused. I notice that the original code only support load .npy file, but the output of the top-down 2D model is always .json file. I modified the data loading functions, I'm not sure whether this change may cause the error.

And also, I just use 1/15 of h36m dataset for evaluation, but I noticed that CPN 2D detections use all of he datasets, around 550K. I'm alse not sure whether this change may cause the error.