scaredyfish / blender-rhubarb-lipsync

Blender Rhubarb Lipsync is an addon for Blender integrating Rhubarb Lip Sync to automatically generate mouth-shape keyframes from a pose library.
187 stars 33 forks source link

Blender 3.0.1 - Phonemes detection problem #44

Closed tonidomi83 closed 11 months ago

tonidomi83 commented 2 years ago

Hi!

First of all, thanks for such an amazing tool, it's a life saver for us animators ;)

I've been trying to rig a character's mouth in Blender as the one explained in this tutorial:

https://www.youtube.com/watch?v=pxy9gkCXPGc

Basically, it's a texture rig driven by a bone. I've setup the pose library and assigned the corresponding visemes in the rhubarb section of the armature, and when I run the script everything runs as supposed. But it seems like the visemes are not dected and applied correctly. The pacing of the speech does work, but it seems as it doesn't use some specific shapes. For example, there are no AI sounds.

I attach a link to a quick screen recording of my process and testing with different files so you can see if anything I do is not correct.

I'm using blender 3.0.1 and version 3.0.2 of your addon. Can you please help me detect the problem?

Thanks again for your effort and hope to hear soon from you,

https://user-images.githubusercontent.com/107185718/172832151-06a67095-60c8-4118-8b0a-ff8c0409e572.mp4

scaredyfish commented 2 years ago

I can't see anything obviously wrong. It seems like Rhubarb is running correctly, perhaps try changing the recogniser from PocketSphinx to Phonetic in the addon settings and see if that helps. You can also open the console (Window->Toggle system console) and see if there are any error messages reported. You should be able to see if the visemes are being detected by Rhubarb and not being shown, or whether Rhubarb itself is not detecting the visemes.

Ultimately, I think this may be a problem with Rhubarb rather than the Blender addon, so you may have to seek support at https://github.com/DanielSWolf/rhubarb-lip-sync