search

met4citizen / TalkingHead

Talking Head (3D): A JavaScript class for real-time lip-sync using Ready Player Me full-body 3D avatars.
MIT License
349 stars 107 forks source link

Creating new pose templates #68

Open lupettohf opened 3 weeks ago

lupettohf commented 3 weeks ago

How cna we create custom poses templates? Doing it through code is possible but a hassle, is there any way to export the poses/animations form blender? We wanted to make a floating bee, so we canno use the default animations.

met4citizen commented 3 weeks ago

A floating bee? That sounds intriguing...

The basics of creating a custom pose template are explained in README Appendix D. And you're right, it would be quite time-consuming to manually type in every bone rotation. Here's a trick I have used:

  1. Download a Mixamo pose FBX file or, in your case, create your own pose using Blender, then export its animation as an FBX file. Ensure that the pose is set in the first animation frame.
  2. Open the local FBX file using the TalkingHead test app: navigate to Avatar | Pose | Folder icon. (If you haven't installed the test app, you can try it here, since no API keys are required for this.)
  3. If your FBX animation is Mixamo/RPM/TalkingHead compatible, the avatar should now be in the specified pose. Open the browser console and enter:
head.getPoseString(head.poseTarget.props)

You'll receive a string like this, which you can copy and paste directly into your custom pose template data structure:

"{'Hips.position':{x:0.01, y:0.942, z:0.018}, 'Hips.rotation':{x:-0.1, y:0.443, z:0.022}, 'Spine.rotation':{x:-0.168, y:-0.159, z:-0.161}, 'Spine1.rotation':{x:0.015, y:-0.031, z:0.042}, 'Spine2.rotation':{x:0.042, y:-0.03, z:0.042}, 'Neck.rotation':{x:0.127, y:-0.019, z:-0.02}, 'Head.rotation':{x:0.338, y:-0.128, z:-0.026}, 'RightShoulder.rotation':{x:1.689, y:-0.157, z:1.844}, 'RightArm.rotation':{x:-0.234, y:0.002, z:-0.402}, 'RightForeArm.rotation':{x:0, y:0, z:-0.685}, 'RightHand.rotation':{x:-1.087, y:-0.084, z:-0.392}, 'RightHandThumb1.rotation':{x:0.257, y:0.069, z:-0.391}, 'RightHandThumb2.rotation':{x:-0.07, y:0.041, z:0.33}, 'RightHandThumb3.rotation':{x:-0.041, y:-0.001, z:-0.013}, 'RightHandIndex1.rotation':{x:-0.171, y:0.002, z:-0.021}, 'RightHandIndex2.rotation':{x:0.059, y:0, z:0.007}, 'RightHandIndex3.rotation':{x:0.059, y:0, z:0.007}, 'RightHandMiddle1.rotation':{x:-0.174, y:0.002, z:-0.021}, 'RightHandMiddle2.rotation':{x:0.048, y:0, z:0.006}, 'RightHandMiddle3.rotation':{x:0.002, y:0, z:0}, 'RightHandRing1.rotation':{x:-0.055, y:0, z:-0.007}, 'RightHandRing2.rotation':{x:0.019, y:0, z:0.002}, 'RightHandRing3.rotation':{x:0.004, y:0, z:0}, 'RightHandPinky1.rotation':{x:0.13, y:0.001, z:0.016}, 'RightHandPinky2.rotation':{x:0.076, y:0, z:0.009}, 'RightHandPinky3.rotation':{x:0.064, y:0, z:0.008}, 'LeftShoulder.rotation':{x:1.704, y:0.404, z:-1.643}, 'LeftArm.rotation':{x:0.601, y:-0.421, z:1.269}, 'LeftForeArm.rotation':{x:0, y:0, z:0.739}, 'LeftHand.rotation':{x:-0.382, y:0.104, z:-0.162}, 'LeftHandThumb1.rotation':{x:0.253, y:-0.098, z:0.311}, 'LeftHandThumb2.rotation':{x:-0.057, y:-0.027, z:-0.259}, 'LeftHandThumb3.rotation':{x:-0.049, y:0.002, z:-0.019}, 'LeftHandIndex1.rotation':{x:-0.068, y:0, z:0.008}, 'LeftHandIndex2.rotation':{x:0.092, y:-0.001, z:-0.011}, 'LeftHandIndex3.rotation':{x:0.116, y:-0.001, z:-0.014}, 'LeftHandMiddle1.rotation':{x:-0.007, y:0, z:0.001}, 'LeftHandMiddle2.rotation':{x:0.118, y:-0.001, z:-0.014}, 'LeftHandMiddle3.rotation':{x:0.177, y:-0.002, z:-0.021}, 'LeftHandRing1.rotation':{x:0.033, y:0, z:-0.004}, 'LeftHandRing2.rotation':{x:0.157, y:-0.001, z:-0.019}, 'LeftHandRing3.rotation':{x:0.218, y:-0.003, z:-0.026}, 'LeftHandPinky1.rotation':{x:0.076, y:0, z:-0.081}, 'LeftHandPinky2.rotation':{x:0.275, y:0.015, z:0.036}, 'LeftHandPinky3.rotation':{x:0.271, y:-0.004, z:-0.032}, 'RightUpLeg.rotation':{x:-0.598, y:0.002, z:3.052}, 'RightLeg.rotation':{x:-1.062, y:0.119, z:0.035}, 'RightFoot.rotation':{x:0.971, y:-0.042, z:0.129}, 'RightToeBase.rotation':{x:0.901, y:0.018, z:-0.001}, 'LeftUpLeg.rotation':{x:-0.117, y:0.059, z:-3.126}, 'LeftLeg.rotation':{x:-0.974, y:0.067, z:-0.043}, 'LeftFoot.rotation':{x:1.309, y:0.137, z:0.038}, 'LeftToeBase.rotation':{x:1.019, y:-0.009, z:0}}"