search

met4citizen / TalkingHead

Talking Head (3D): A JavaScript class for real-time lip-sync using Ready Player Me full-body 3D avatars.
MIT License
339 stars 105 forks source link

a-frame demo #53

Closed Roxcsas closed 3 months ago

Roxcsas commented 3 months ago

Hi, im trying to make a porting in angular and a-frame (framework built on top of three.js) but im not sure if i can just use the THREE lerp with the frame times of TalkingHead for the lip sync animations

code: https://github.com/Roxcsas/aframe-talking-avatar-ia

this is the result: https://streamable.com/flpvvh

met4citizen commented 3 months ago

Very interesting. Thank you for sharing.

I don't see any problem with using the THREE lerp. It does more or less the same thing as my valueAnimationSeq method. One difference here is that you use a linear animationProgress with lerp, whereas I use a sigmoid function to ease in and out. If you want to experiment with this, you can check out my sigmoidFactory method and do the following:

this.easing = this.sigmoidFactory(5);
...
let animationProgress = Math.max( ...
animationProgress = myComponent.easing(animationProgress);

However, I don't know if this makes much difference and there is no single right way to do this. Mostly, it is just trial-and-error and some fine-tuning, and in the end, it all comes down to what looks most natural lip movement to you. Based on the video clip, your code is already working very well.