Creating new phonemes Table

[Samdaught]

I tried to create a table based on IPA for Russian, but it didn't work. can anyone help/tell me how to create it? The addon recognises speech (it can be seen in the blender3d console), but does not create animations. the problem is in the table.

[blackears]

Make sure to match the symbols being output by Gruut, which may not be exact matches. Also, if you use the English table, does the plugin produce animations?

[Samdaught]

I've tried with English speech and a table (like yours) - the character works perfectly. But if i replace either the text or the table with Russian text - everything stops working. I can see in the blender console that it recognises speech word by word.

[blackears]

If you can post an example file, I can take a look.

[Samdaught]

you mean table?

[blackears]

The audio file that you're trying to translate and the table that you're trying to use. If you can post the animated rig you're trying to use, that could help too.

[blackears]

Basically everything that I'd need to try and run the same process on my machine.

[Samdaught]

I'm pretty sure I haven't figured out how to reconfigure the table to another language. so I'll be glad for any hints.

Here are the files: different variations of russian tables, English and Russian speech, project.

https://drive.google.com/file/d/11azn0c7VD28z-uddOfGfMBmnXh6_E1c7/view?usp=drive_link

[Samdaught]

I forgot to open access, sorry, it'll work now.

[blackears]

Looks like the issue is that Gruut is returning empty data for phonemes for the Russian language. I'm not sure why - Russian is supposed to be a supported language. I've posted an issue on their forum - hopefully they will get back.

[Samdaught]

Thank you! I'll look forward to it.

[blackears]

You'll need to install a Russian language pack. You'll also need to download the latest version of parrot - I needed to make some changes to handle the different output Gruut is providing.

./python.exe -m pip install --upgrade gruut-lang-ru

[Samdaught]

Thanks! I'll try it soon.

[Samdaught]

I updated gruut, put 1.0.2 version of the plugin. The model still works fine with English speech and table. but it doesn't work with Russian. even if I just leave the English table and turn on the Russian audio track - empty keyframes are created. The same when I plug in the Russian phoneme table.

[blackears]

Looks like the problem is that some of your mouth poses only keyframe part of a compound attribute. This works on the test file you sent me. Please try it and let me know if it works. Also, the phoneme table you've constructed needs to use the symbols printed to the console - most of the IPA symbols you're using are not the ones Gruut needs.

parrotLipsync.zip

[Samdaught]

Thanks! After I updated the library, the lipsync worked: selecting Russian audio track and English phoneme - now all keyframes are set correctly and speech works. The Russian table doesn't work, but I'm sure I just filled it in wrong. Could you send me a link to the phoneme site? I searched, but nothing came up. For now I will use the broadcast with English phonemes, they seem to work well.

[blackears]

The best way to check phonemes it to examine the console output window when you run Parrot. It will print out a Missing Phoneme warning each time it encounters a phoneme that you've not defined in your table. I noticed your table uses some Cyrillic characters where Gruut seems to be expecting roman ones. You might just want to double check that your custom table is using the symbols Gruut is expecting.

[Samdaught]

Thanks!