search

CSTR-Edinburgh / merlin

This is now the official location of the Merlin project.
http://www.cstr.ed.ac.uk/projects/merlin/
Apache License 2.0
1.31k stars 440 forks source link

Creating synthesizer for new languages in Merlin #429

Open arif334 opened 5 years ago

arif334 commented 5 years ago

I want to use Merlin to synthesize voices in Bangla (Bengali). Can anyone guide me to the steps of creating new languages in Merlin? Besides the data-set, what language-specific tasks do we need to perform? I believe the modifications must be done in the front-end (festival). Any step-by-step instructions, or links to any existing online tutorials would be appreciated. Thanks in advance.

rhoposit commented 5 years ago

Hi Arif - got your email. Thanks for reaching out to me directly, however I am not an expert in the Merlin package here. So perhaps others will be able to give you better and detailed info. It looks like other issues on this repository report that it can be done in other languages.

Yes you need to set up some front-end pre-processing that is specific to your language. For example, you'll want to have a part-of-speech tagger trained up (I would recommend StanfordNLP tagger) to identify content words. You'll need some lexicons also in the Bangla language. You can do this with G2P (you'll need to train it for your language) or if you already have the lexicons that is fine but you'll want some support for OOV words. Also take a look at your questions file to make sure that the questions matches the pre-processing. Watch out for utf-8 codepoints - I would recommend using SAMPA instead of IPA in your lexicons. Then you will need to use your front-end processing and questions file to generate your label files.

arif334 commented 5 years ago

@rhoposit Thank you for your reply. It'd give me some initial ideas. I already have some of the tools (g2p, lexicon). Hopefully, I can get started from this point. Thanks again.

pasindud commented 5 years ago

FYI there are some resources here [1], also there is a rough voice here [2]

1 - https://github.com/googlei18n/language-resources/tree/master/bn/merlin 2 - http://tinyurl.com/voice-builder

arif334 commented 5 years ago

Thank you @pasindud for pointing out the resources. Can I use my own data to create a festival voice and to train merlin models? What changes do I need to make?