search

steveway / papagayo-ng

Papagayo is a lip-syncing program designed to help you line up phonemes (mouth shapes) with the actual recorded sound of actors speaking. Papagayo makes it easy to lip sync animated characters by making the process very simple - just type in the words being spoken (or copy/paste them from the animation's script), then drag the words on top of the sound's waveform until they line up with the proper sounds.
http://steveway.github.io/papagayo-ng/
18 stars 3 forks source link

Missing conversion error #19

Closed Hunanbean closed 3 years ago

Hunanbean commented 3 years ago

Missing conversion for: ɻ Traceback (most recent call last): File "O:\papagayo-ng-allosaurus\LipsyncFrameQT.py", line 432, in on_open self.open(file_path) File "O:\papagayo-ng-allosaurus\LipsyncFrameQT.py", line 468, in open self.doc.auto_recognize_phoneme() File "O:\papagayo-ng-allosaurus\LipsyncDoc.py", line 850, in auto_recognize_phoneme word.text = "".join(letter["phoneme"] for letter in word_chunk) TypeError: sequence item 0: expected str instance, NoneType found

I was finally able to give this a try. It looks very cool! I am not sure it is working as expected though. I was only able to get 1 sound file to work with it, and the auto breakdown was nothing near what i expected. It seemed the breakdown did not remotely match the spoken words. The sound file i was able to get recognized was of low quality though. That may be the issue. Thanks!

steveway commented 3 years ago

Oh, I see what the problem is. When creating the words I forgot to handle None phonemes. I added some handling for that. I also added that "ɻ" phoneme to my ipa to cmu conversion, it's pretty much just an "R". If you're having problems loading soundfiles then I would recommend checking if Pydub and FFMPEG is installed. On Windows use the Download FFMPEG Button in Papagayo-NG. Actually PyDub seems to now need ffprobe from FFMpeg, I already changed it so it will also extract that. But now I've added some logic to check if it is present and also you can delete the ffmpeg binaries from the settings. After you have downloaded ffmpeg, it is best to restart Papagayo-NG.

steveway commented 3 years ago

If the result then is still not good can you send an example which I can test? Because with the Papagayo Tutorial files, even the spanish ones, the results are really good. (The conversion to other phonemesets is not good yet, so best keep it to CMU39 for now.)

Hunanbean commented 3 years ago

Audio files that would not load yesterday, when i was also having the false drag and drop problem, are loading today. I am retesting now. I thought i had reloaded after the FFMPEG, but perhaps not.

Thank you

Hunanbean commented 3 years ago

Oh, that is so cool!! I forgot that often times, i have to load, close, reload for the software to function properly after changes. It is working now, and the error is gone with the fixes you implemented. That is really impressive! seems at least 95%+ accuracy (just observation, I mean perceived results, not a full analysis). Thank you so much for knowing how, and spending the time and effort to share this with us! It is sincerely appreciated. When this gets into the main version, i expect usage to skyrocket. I intend to post a video and start spreading the word again when this gets into main. Also, perhaps not the best place to discuss it, but it came to my attention that there are pre-compiled versions available to patreons? If this is infact the case, i will mention that in the video.