MontrealCorpusTools / Montreal-Forced-Aligner

Command line utility for forced alignment using Kaldi
https://montrealcorpustools.github.io/Montreal-Forced-Aligner/
MIT License
1.29k stars 243 forks source link

Understanding MFA phoneset #584

Open bluemailbox opened 1 year ago

bluemailbox commented 1 year ago

Is your feature request related to a problem? Please describe. I'm trying to map the MFA phoneset to images representing mouths making that sound, but I need to find reference for what all the unicoded characters sound like. In specific I am using the french dict/acoustics in MFA format, but when i generate a phoneme list I have no way of interpreting what the phonemes actually sound like. Looking for a reference sheet so i can loosely map these to phoneme images to generate a lip sync'd animation

Describe the solution you'd like I'd like to have a way to convert these phonemes into a simpler structure like ARPA, so I can pick through them and map them to mouth images. These are the characters handed back by MFA alignment: ø ŋ œ ɑ ɑ̃ ɔ ɔ̃ ə ɛ ɛ̃ ɟ ɡ ɥ ɲ ʁ ʃ ʎ ʒ that i do not understand.

License: CC BY 4.0

Describe alternatives you've considered Our video generation software is hard coded to work with ARPA formatted phonemes, as they are regular characters i can decipher. We considered re-writing the program but that will only work for one language at a time, but the end goal is to have as much language support as possible.

Additional context Add any other context or screenshots about the feature request here.

mmcauliffe commented 1 year ago

The International Phonetic Alphabet (IPA) charts for the dictionary are here: https://mfa-models.readthedocs.io/en/latest/dictionary/French/French%20MFA%20dictionary%20v2_0_0a.html, and I'd recommend reading up about French phonology to get a handle on the symbols with examples, but I would say that ARPA is not going to be very helpful for French. IPA can be represented in xsampa if you're more familiar with that (IPA-XSAMPA converter).