shivammehta25 / Matcha-TTS

[ICASSP 2024] 🍵 Matcha-TTS: A fast TTS architecture with conditional flow matching
https://shivammehta25.github.io/Matcha-TTS/
MIT License
747 stars 95 forks source link

Issues with interrogative sentences when using ljspeech model #79

Closed orenstuf closed 5 months ago

orenstuf commented 5 months ago

Hello,

Here are some issues with interrogative sentences:

Can you give me some guidelines on how to train matcha to overcome these issues?

Thanks a lot

shivammehta25 commented 5 months ago

Hi @orenstuf,

The LJ Speech checkpoint is only trained on LJ Speech corpus. Being a read speech corpus it doesn't really have a lot of variation or interrogative sentences for the model to learn.

The only mitigation for this will be to improve the dataset. Or try fine-tuning the model to your voice. Record 20 such sentences and fine-tune and see if the model learns something from it.

Regards, Shivam