search

morevnaproject-org / papagayo-ng

Papagayo is a lip-syncing program designed to help you line up phonemes (mouth shapes) with the actual recorded sound of actors speaking. Papagayo makes it easy to lip sync animated characters by making the process very simple - just type in the words being spoken (or copy/paste them from the animation's script), then drag the words on top of the sound's waveform until they line up with the proper sounds.
233 stars 51 forks source link

Allosaurus integration #94

Closed steveway closed 3 years ago

steveway commented 3 years ago

This Pull request integrates Allosaurus into Papagayo-NG. It uses the CMU_39 phoneme set. I've also added some simple logic to convert between different phoneme sets. Just select the set you want and press the action at the top to convert. (Maybe move the action to a button on the top toolbar instead.) The results are not really good, it would be best to have a handcrafted dictionary for every possible set conversion. We are only supporting 4 sets at the moment:

This change also adds a new Settings Dialog, a few settings can already be changed with it.

At the moment similar to Rhubarb before it will try to parse the sound file using allosaurus when you open a single sound file. Technically it should also fall back to Rhubarb if Allosaurus fails, not sure if the logic is correct. But we should add some buttons to initiate the automatic decoding via Allosaurus and Rhubarb manually.

A next feature which will be helpful would be to move phonemes/words/phrases between voices. Currently all the object belong to one voice when recognized automatically. So if we have different speakers we want to switch some parts between them. I've already added some automatic separation between possible words based on the size of gaps between phonemes to make selecting groups easier.

aziagiles commented 1 year ago

@steveway Hello Steve. I used to use the Papagayo-NG Allosaurus version, but recently I realize you are on another version called the PySide6. For usability in 2023, which will you recommend we use? Allosaurus version or Pyside 6 version?

steveway commented 1 year ago

Hi, the PySide6 version still has at least one big bug which needs to be fixed. But this seems to be an underlying bug in PySide6 or even QT6: https://github.com/morevnaproject-org/papagayo-ng/issues/117 When this is fixed by them or if I make a workaround then that version should be better for the future. I will then also merge that with this main project and create a new release.