How can we get evaluation results?

[ichbill]

Hi, I want to ask about the code for evaluation part. How can we get results as well as evaluation metrics such as MPJPE and PA-MPJPE?

[penincillin]

You can calculate the MPJPE and PA-MPJPE according to the definition.

[ichbill]

Hi, currently I'm using agora evaluation tool to get metrics, because I saw your model on their leaderboard. However I got stacked when preparing predictions.

I'm using SMPLX model, and agora evaluation requires joints(2d projection joints location in the image), verts(3d vertices in camera coordinates), and allSmplJoints3d(3d joints in camera coordinates). I've got joints and verts from pred_joints_img and pred_vertices_img, but could you tell how do we get allSmplJoints3d from Frankmocap output?

[penincillin]

@ichbill For joints (2D keypoints), you can use pred_joints_img via taking the first two dims (x and y). For verts, you can use pred_verts_smpl. For allSmplJoints3d, you can use pred_joints_smpl.
Please be noted that pred_joints_smpl and pred_verts_smpl are not in camera space. Therefore, you may align the predicted and gt joints by root or Procrustes analysis before calculating metrics such as MPJPE.

[ichbill]

@penincillin Thank you for the reply! However when I tried to run xvfb-run -a python -m demo.demo_bodymocap --input_path ./sample_data/agora --out_dir ./mocap_output/ --save_bbox_output --save_pred_pkl --save_mesh --no_display --use_smplx --renderer_type opengl, pred_joints_smpl was not stored in the pkl file. Is there anything I missed? I also checked "/bodymocap/body_mocap_api.py", but didn't find pred_joints_smpl either. Could you tell how can I get pred_joints_smpl?

[penincillin]

@ichbill I just checked. The pred_joints_smpl is pred_joints_vis in L103.