Step-by-step adding foreign words to ViSV2TTS

[drlor2k]

hello @v-nhandt21 , how are you? I have some questions can you guide me!

Problem:

1. Goal: the model can pronounce some English words in the training set.
2. Premise: I trained the model on a pure Vietnamese dataset (model A).
3. My approach: I will use a small dataset (about 2 hours) that has mixed English and Vietnamese to finetuning model A into model B to achieve 1. Goal.

Questions:

1. Is the above approach feasible?
2. What steps do I need to take for this approach?

I found Viphoneme on github has English support, but this repo doesn't!

Thanks for taking the time to reply!

[v-nhandt21]

Hi @drlor2k,

To handle Foreign languages, as my knowledge we have two main approaches:

• Convert both Vietnamese and English to IPA or some unified grapheme, in viphoneme I use "eng_to_ipa" library, but I am not sure it would work in all cases, and I dont have detailed experiments on how much English data is enough!

• The simpler approach is to use a dictionary in which each word is mapped to Vietnamese syllable pronunciation

p/s: you can take a look for this for more approach: https://github.com/tuanh123789/AdaSpeech/tree/main/G2P

p/s 2: maybe there are some multilingual end2end models that can speak all of this without phoneme control by using a unified tokenizer :))