Open Dipankar1997161 opened 1 year ago
This is because SMPL only have limited joints for face and hand, you can see SMPL keypoints here
Nose is not included in smpl joint set (which only have 24 joints), convert_kps only converts keypoints with similiar definitions.
Considering the source of joints annotations, I suggest two ways:
Hello @Wei-Chen-hub, Thanks for the clarification.
The smpl data I have generated has 24 joints, which is the standard used. For the first method, is the full smpl regression possible through mmhuman3d? if yes, can you refer it to me.
I was thinking of the second method, but couldn't really find a proper source to read and try it out. I did try one directly with coco, but it turned out incorrectly.
Looking forward to hear from you
For joints regression from SMPL parameters, you can see the official SMPL/SMPLX repo.
Alternatively, in MMHuman3D you can do this through several steps:
from mmhuman3d.models.body_models.builder import build_body_model
smpl_model = build_body_model(dict( type='SMPL', keypoint_src='smpl_45', keypoint_dst='smpl_45', model_path='data/body_models/smpl', # modify this path to your SMPL model folder gender='neutral', num_betas=10, use_pca=False, batch_size=1)).to(self.device)
- Input parameters and get 3d joints
output = smpl_model(parameters_dict, return_joints=True) # smpl parameters keypoints_3d = output['joints'].detach().cpu().numpy()
The input parameters are basically same as the **official inplementation**.
The data I need is a mediapipe key point. Could you tell me how to do a convention with a mediapipe keypoint?
@Wei-Chen-hub Is it right to write the build_body_model in estimate_smpl.py?
@Dipankar1997161 Can I know how to convert keypoints to mediapipes?
Hello @Wei-Chen-hub,
I used the keypoint convention for smpl to coco. There are certain missing joints after conversion, particularly the face keypoints.
Is it possible to recover "nose" keypoint after the conversion? That's the most important one for me. here is the example result![Screenshot_20230807_155137](https://github.com/open-mmlab/mmhuman3d/assets/85514219/5bb5c068-d2e1-4cd2-bcb3-c645bb273431)
I will wait for your response