How to train a Chinese and English hybrid acoustic model by using MFA?

MontrealCorpusTools / Montreal-Forced-Aligner

Command line utility for forced alignment using Kaldi

https://montrealcorpustools.github.io/Montreal-Forced-Aligner/

MIT License

1.31k stars 244 forks source link

How to train a Chinese and English hybrid acoustic model by using MFA? #733

Open Raise-me-up opened 8 months ago

Raise-me-up commented 8 months ago

Hi, all

I have a request to generate the alignments for the Chinese and English hybrid dataset, but the pretrained model is either pure Chinese or English one. Therefore, I have to train my own model. However, I can't find any tutorial. I don't know which phone set is suitable for me, and how to make a dictory, and so on. Any useful advice will be grateful. Thanks!

NataliaShmueli commented 8 months ago

Hey! There are multiple ways you could do this! The easiest way, in my opinion, would be using IPA for both. MFA can link to a dictionary-per-speaker model. You could use a Mandarin dictionary and an English dictionary and then use this methodology: https://montreal-forced-aligner.readthedocs.io/en/latest/user_guide/dictionary.html#per-speaker-dictionaries

Mind you, this should also improve the model for each other.