search

huawei-noah / Speech-Backbones

This is the main repository of open-sourced speech technology by Huawei Noah's Ark Lab.
557 stars 115 forks source link

Finetuning a Grad-TTS model on a small dataset? #21

Open godspirit00 opened 2 years ago

godspirit00 commented 2 years ago

Hi! Thanks for the great work! I have a small dataset of ~2 hours. How do I finetune a Grad-TTS model on it? Thanks!

ilyalasy commented 1 year ago

Hi, same question, is it possible?

li1jkdaw commented 1 year ago

Hi, @godspirit00, @Misterion777! Yes, it is possible. You can either start from our checkpoint trained on LibriTTS and then fine-tune it on your voice (check #8 for more details), or train Grad-TTS on some large multi-speaker dataset (check #23 for details) and then fine-tune it. If your dataset is in English, then the first option may be the easiest (and around 10 minutes of the target voice will be perhaps enough). If the language is different, then the second may be better (i.e. you can find some large open-source dataset in this language, train Grad-TTS on it, and then fine-tune it on your target voice). In any case, don't forget to use either the pre-trained universal HiFi-GAN vocoder or fine-tune it to your voice for better sound quality.