Vegetebird / MHFormer

[CVPR 2022] MHFormer: Multi-Hypothesis Transformer for 3D Human Pose Estimation
MIT License
521 stars 81 forks source link

After estimating 19 joints, the result is strange. #100

Closed champ5869 closed 1 year ago

champ5869 commented 1 year ago

Hello. author. I am in the process of extracting 19 joint points including the toe of h36m.

order is

  1. Convert data_2d_h36m_gt.npz from 17 to 19 joints
    • Obtained 32 2d joint points with camera_to_pixel() after world_to_camera() in data_3d_h36m.npz. (Using camera internal/external parameters)
    • Keypoints used: ['HIP', 'R-HIP', 'R-KNEE', 'R-ANKLE', 'R-SOLE', 'R-TOE', 'L-HIP', 'L-KNEE', 'L- ANKLE', 'L-SOLE', 'L-TOE', 'NECK', 'HEAD', 'L-SHOULDER', 'L-ELBOW', 'L-WRIST', 'R-SHOULDER', 'R-SHOULDER' ELBOW', 'R-WRIST']
  2. Learning with converted data_2d_h36m_gt.npz and data_3d_h36m.npz
  3. Since HRnet extracts only 17 joint points, after extracting 26 joint points using Alphapose, 19 joint points including BigToe are re-extracted.
    • Keypoints used: {5, "LShoulder"}, {6, "RShoulder"}, {7, "LElbow"}, {8, "RElbow"}, {9, "LWrist"}, {10, "RWrist"}, {11 , "LHip"}, {12, "RHip"}, {13, "LKnee"}, {14, "Rknee"}, {15, "LAnkle"}, {16, "RAnkle"}, {17, " Head"}, {18, "Neck"}, {19, "Hip"}, {20, "LBigToe"}, {21, "RBigToe"}, {24, "LHeel"}, {25, "RHeel" }
  4. 2D to 3D lifting with MHForer

The result is the video below.

https://user-images.githubusercontent.com/53414128/235810795-125191a2-c321-4cd3-9ba7-a6674b908f6f.mp4

https://user-images.githubusercontent.com/53414128/235810703-19a25d97-cf00-4951-aaa9-3d3bdee38b96.mp4

The angle of the heel and toe is strange. Can you tell me why?

srn-source commented 1 year ago

Could you recommend me for github repo because i want to do the same thing as you?