is there [bvh to SMPL] ??

[sunwonlikeyou]

I tried to convert bvh to SMPL using your method. As I know rotation representation of bvh is euler. so i convert euler-> axis angle per each joint. But i couldnt get the desired smpl pose. Do you know how to do it?

[CalciferZh]

I guess this is because you are not using global rotation but parent-relative rotation.

[sunwonlikeyou]

Do you mean i have to use global rotation?? but as i know euler of bvh and smpl axis-angle representations are all parent- relative rotation. Then do i have to convert relative rotations to global rotation both bvh euler and smpl???

"relative euler -> global euler -> global axis-angle -> relative axis-angle(SMPL)" <- Is that right???
or
rotation_matrix_to_axis_angle(axisangle2rotmat(parent_axis_angle)*(parent rotation*child rotation*inv(parent rotation))) ??

[SlimeVRX]

Hi @sunwonlikeyou

Can you share with me the conversion from BVH to SMPL? I found a very cool dataset, I want to convert to SMPL+H

BEAT: A Large-Scale Semantic and Emotional Multi-Modal Dataset for Conversational Gestures Synthesis. Page: https://pantomatrix.github.io/BEAT/

[sunwonlikeyou]

I'm sorry I don't have those SMPLs.. If you want get it, you may need template bvh of same kinematics. Then you can covert those bone angles.

[SlimeVRX]

Hi @sunwonlikeyou

I have selected template bvh of same kinematics.

skeleton_BEAT = {'Hips': 0, 'Spine': 1, 'Spine1': 2, 'Spine2': 3, 'Spine3': 4, 'Neck': 5, 'Neck1': 6, 'Head': 7, 'HeadEnd': 8, 'RightShoulder': 9, 'RightArm': 10, 'RightForeArm': 11, 'RightHand': 12, 'RightHandMiddle1': 13, 'RightHandMiddle2': 14, 'RightHandMiddle3': 15, 'RightHandMiddle4': 16, 'RightHandRing': 17, 'RightHandRing1': 18, 'RightHandRing2': 19, 'RightHandRing3': 20, 'RightHandRing4': 21, 'RightHandPinky': 22, 'RightHandPinky1': 23, 'RightHandPinky2': 24, 'RightHandPinky3': 25, 'RightHandPinky4': 26, 'RightHandIndex': 27, 'RightHandIndex1': 28, 'RightHandIndex2': 29, 'RightHandIndex3': 30, 'RightHandIndex4': 31, 'RightHandThumb1': 32, 'RightHandThumb2': 33, 'RightHandThumb3': 34, 'RightHandThumb4': 35, 'LeftShoulder': 36, 'LeftArm': 37, 'LeftForeArm': 38, 'LeftHand': 39, 'LeftHandMiddle1': 40, 'LeftHandMiddle2': 41, 'LeftHandMiddle3': 42, 'LeftHandMiddle4': 43, 'LeftHandRing': 44, 'LeftHandRing1': 45, 'LeftHandRing2': 46, 'LeftHandRing3': 47, 'LeftHandRing4': 48, 'LeftHandPinky': 49, 'LeftHandPinky1': 50, 'LeftHandPinky2': 51, 'LeftHandPinky3': 52, 'LeftHandPinky4': 53, 'LeftHandIndex': 54, 'LeftHandIndex1': 55, 'LeftHandIndex2': 56, 'LeftHandIndex3': 57, 'LeftHandIndex4': 58, 'LeftHandThumb1': 59, 'LeftHandThumb2': 60, 'LeftHandThumb3': 61, 'LeftHandThumb4': 62, 'RightUpLeg': 63, 'RightLeg': 64, 'RightFoot': 65, 'RightForeFoot': 66, 'RightToeBase': 67, 'RightToeBaseEnd': 68, 'LeftUpLeg': 69, 'LeftLeg': 70, 'LeftFoot': 71, 'LeftForeFoot': 72, 'LeftToeBase': 73, 'LeftToeBaseEnd': 74}

skeleton_SMPL_X = {'pelvis': 0, 'left_hip': 1, 'left_knee': 2, 'left_ankle': 3, 'left_foot': 4, 'right_hip': 5, 'right_knee': 6, 'right_ankle': 7, 'right_foot': 8, 'spine1': 9, 'spine2': 10, 'spine3': 11, 'neck': 12, 'head': 13, 'jaw': 14, 'left_eye_smplhf': 15, 'right_eye_smplhf': 16, 'left_collar': 17, 'left_shoulder': 18, 'left_elbow': 19, 'left_wrist': 20, 'left_index1': 21, 'left_index2': 22, 'left_index3': 23, 'left_middle1': 24, 'left_middle2': 25, 'left_middle3': 26, 'left_pinky1': 27, 'left_pinky2': 28, 'left_pinky3': 29, 'left_ring1': 30, 'left_ring2': 31, 'left_ring3': 32, 'left_thumb1': 33, 'left_thumb2': 34, 'left_thumb3': 35, 'right_collar': 36, 'right_shoulder': 37, 'right_elbow': 38, 'right_wrist': 39, 'right_index1': 40, 'right_index2': 41, 'right_index3': 42, 'right_middle1': 43, 'right_middle2': 44, 'right_middle3': 45, 'right_pinky1': 46, 'right_pinky2': 47, 'right_pinky3': 48, 'right_ring1': 49, 'right_ring2': 50, 'right_ring3': 51, 'right_thumb1': 52, 'right_thumb2': 53, 'right_thumb3': 54}

mapping = {}

mapping['pelvis'] = 'Hips'  #BVH
mapping['left_hip'] = 'LeftUpLeg'
mapping['right_hip'] = 'RightUpLeg'
mapping['spine1'] = 'Spine'
mapping['left_knee'] = 'LeftLeg'
mapping['right_knee'] = 'RightLeg'
mapping['spine2'] = 'Spine2'
mapping['left_ankle'] = 'LeftForeFoot'
mapping['right_ankle'] = 'RightForeFoot'
mapping['spine3'] = 'Spine3'
mapping['left_foot'] = 'LeftToeBase'
mapping['right_foot'] = 'RightToeBase'
mapping['neck'] = 'Neck'
mapping['left_collar'] = 'LeftShoulder'
mapping['right_collar'] = 'RightShoulder'
mapping['head'] = 'Head'
mapping['left_shoulder'] = 'LeftArm'
mapping['right_shoulder'] = 'RightArm'
mapping['left_elbow'] = 'LeftForeArm'
mapping['right_elbow'] = 'RightForeArm'
mapping['left_wrist'] = 'LeftHand'
mapping['right_wrist'] = 'RightHand'
mapping['jaw'] = 0
mapping['left_eye_smplhf'] = 0
mapping['right_eye_smplhf'] = 0
mapping['left_index1'] = 'LeftHandIndex1'
mapping['left_index2'] = 'LeftHandIndex2'
mapping['left_index3'] = 'LeftHandIndex3'
mapping['left_middle1'] = 'LeftHandMiddle1'
mapping['left_middle2'] = 'LeftHandMiddle2'
mapping['left_middle3'] = 'LeftHandMiddle3'
mapping['left_pinky1'] = 'LeftHandPinky1'
mapping['left_pinky2'] = 'LeftHandPinky2'
mapping['left_pinky3'] = 'LeftHandPinky3'
mapping['left_ring1'] = 'LeftHandRing1'
mapping['left_ring2'] = 'LeftHandRing2'
mapping['left_ring3'] = 'LeftHandRing3'
mapping['left_thumb1'] = 'LeftHandThumb1'
mapping['left_thumb2'] = 'LeftHandThumb2'
mapping['left_thumb3'] = 'LeftHandThumb3'
mapping['right_index1'] = 'RightHandIndex1'
mapping['right_index2'] = 'RightHandIndex2'
mapping['right_index3'] = 'RightHandIndex3'
mapping['right_middle1'] = 'RightHandMiddle1'
mapping['right_middle2'] = 'RightHandMiddle2'
mapping['right_middle3'] = 'RightHandMiddle3'
mapping['right_pinky1'] = 'RightHandPinky1'
mapping['right_pinky2'] = 'RightHandPinky2'
mapping['right_pinky3'] = 'RightHandPinky3'
mapping['right_ring1'] = 'RightHandRing1'
mapping['right_ring2'] = 'RightHandRing2'
mapping['right_ring3'] = 'RightHandRing3'
mapping['right_thumb1'] = 'RightHandThumb1'
mapping['right_thumb2'] = 'RightHandThumb2'
mapping['right_thumb3'] = 'RightHandThumb3'

I have converted to axis-angle but the movement is not as expected. Can you help me?

Sample dataset include: BEAT.bvh and SMPL_X.bvh

sample.zip

Please help me!

[sunwonlikeyou]

They provide conversion pipeline which converts SMPL result to FBX(animation data), maybe you can utilize this. https://github.com/mkocabas/VIBE

[SlimeVRX]

Hi @sunwonlikeyou

I need to convert from BVH to SMPL, not SMPL to FBX