SortAnon / ControllableTalkNet

A web app that lets you play around with TalkNet models
GNU Affero General Public License v3.0
121 stars 48 forks source link

ControllableTalkNet for another language #8

Open Patryk6M opened 2 years ago

Patryk6M commented 2 years ago

In Poland, voice cloning AI is very popular, but tacotron2 does not allow adding emotions and singing. TalkNet technology seems to be brilliant, I would like to make a version for Polish language, but I don't have much IT knowledge and I need some light help.

I have been practicing for a week a 30-hour Polish audiobook "The Doll" on this Colab notepad: https://colab.research.google.com/drive/1VqSWRU1H3KIU6au_ojOGFtU0HQPUFa6t

However, despite quite a bit of training, it still twists words a lot. I have discovered that the problem is not necessarily with the model, but perhaps with the synthesis notebook, which is tailored exclusively for English: https://colab.research.google.com/drive/1aj6Jk8cpRw7SsN3JSYCv57CrR6s0gYPB Everything you type characters into the generator field is converted to English ARPAbet. Polish_TalkNet Is it possible to disable this conversion? Alternatively, is it possible to adapt this ARPAbet for the Polish language? But here there is a problem, because in Polish there are consonants which are not present in English, for example "ć", "ś", "ń", "ź".