met4citizen / TalkingHead

Talking Head (3D): A JavaScript class for real-time lip-sync using Ready Player Me full-body 3D avatars.
MIT License
350 stars 108 forks source link

Makeup lost? #20

Closed JPhilipp closed 7 months ago

JPhilipp commented 7 months ago

Hi! My avatar has lost their lipstick when I include it. (Note the lipstick wasn't a new inclusion, it was added originally when setting up the avatar and getting its id.)

I kind of like the non-make-up version better so it's not a big deal, but wanted to let you know here. I used the mp3.html as a starting point for inclusion. Inclusion shown as square at the top in below screenshot.

makeup

cheers!

met4citizen commented 7 months ago

Based on the differences in hair tone and reflections, I believe this is simply a matter of varying lighting conditions. You can modify the lighting with the setLighting method. The scene has three lights: ambient, directional, and spotlight, and you can adjust their properties such as intensity, angle, and color. See README for details.

JPhilipp commented 7 months ago

I was able to confirm that it's not the lighting which loses the lipstick, but rather, it's the parameters added to the glb URL which when downloaded lose that texture.

// The standard glb url provided by ReadyPlayerMe, it has makeup texture https://models.readyplayer.me/[some-id].glb

// The glb url as used by the avatar loading sample (the download is also a few MB larger) https://models.readyplayer.me/[some-id].glb?morphTargets=ARKit,Oculus+Visemes,mouthOpen,mouthSmile,eyesClosed,eyesLookUp,eyesLookDown&textureSizeLimit=1024&textureFormat=png

I prefer the non-lipstick version you turn it into, and wasn't otherwise able to get rid of the strong lips color even in the non-makeup setting in ReadyPlayerMe configurator when downloading the glb from their URL, so this is a nice workaround!

met4citizen commented 7 months ago

I just tested this myself and you are right. It is the textureFormat parameter. If I download the avatar with the value png, the lips look much lighter (a lot less red) than when using jpeg or webp. I can see the difference also in Blender, so the problem lies with the Ready Player Me service. I could report this to their support, but actually, I also prefer the less bright color. Thanks for letting me know.