Why the body_pose parameter of the model in the results is 21*3

[ireneisme]

Thanks very much for sharing this wonderful project! I am new in this field and ECON is my first try to generate an avatar from single image. After runing the code, I found the parameter of the model input is somewhat different with my understanding about SMPLX.

I have printed the shape of each variable showned below. What I am confused about is the number of body_pose. According to the paper, I think it should be 24 joints times 3 dimensions which contains 2 more joints. Can you pls exlain what are missing or what is wrong with my understanding of this question?

betas :  torch.Size([1, 200])
body_pose :  torch.Size([1, 21, 3])
global_orient :  torch.Size([1, 1, 3])
transl :  torch.Size([1, 3])
expression :  torch.Size([1, 50])
jaw_pose :  torch.Size([1, 1, 3])
left_hand_pose :  torch.Size([1, 15, 3])
right_hand_pose :  torch.Size([1, 15, 3])
scale :  torch.Size([1, 1])

One more thing, could you also provide some suggestions about how to convert this to the SMPL model one? Thanks a lot!

[YuliangXiu]

Thanks for asking

21 = 25-1(pelvis)-2(left eye, right eye)-1(jaw)

[ireneisme]

Really appreciate your quick and precise reply😄 Sry to bother u again @YuliangXiu , but I am still wondering how I can get the left & right eye pose parameter from the .npy file?

 smpl_info = {
                    "betas":
                    optimed_betas[idx].detach().cpu().unsqueeze(0),
                    "body_pose":
                    rotation_matrix_to_angle_axis(optimed_pose_mat[idx].detach()
                                                 ).cpu().unsqueeze(0),
                    "global_orient":
                    rotation_matrix_to_angle_axis(optimed_orient_mat[idx].detach()
                                                 ).cpu().unsqueeze(0),
                    "transl":
                    optimed_trans[idx].detach().cpu(),
                    "expression":
                    data["exp"][idx].cpu().unsqueeze(0),
                    "jaw_pose":
                    rotation_matrix_to_angle_axis(data["jaw_pose"][idx]).cpu().unsqueeze(0),
                    "left_hand_pose":
                    rotation_matrix_to_angle_axis(data["left_hand_pose"][idx]).cpu().unsqueeze(0),
                    "right_hand_pose":
                    rotation_matrix_to_angle_axis(data["right_hand_pose"][idx]).cpu().unsqueeze(0),
                    "scale":
                    data["scale"][idx].cpu(),
                }

And I have checked how to generate a SMPLX model:

        if create_leye_pose:
            if leye_pose is None:
                default_leye_pose = torch.zeros([batch_size, 3], dtype=dtype)
            else:
                default_leye_pose = torch.tensor(leye_pose, dtype=dtype)
            leye_pose_param = nn.Parameter(default_leye_pose, requires_grad=True)
            self.register_parameter("leye_pose", leye_pose_param)

So I should just set both eye para to 0 because there is no such attribution loaded from our .npy file? Thx very much!