search

CorentinJ / Real-Time-Voice-Cloning

Clone a voice in 5 seconds to generate arbitrary speech in real-time
Other
51.54k stars 8.64k forks source link

emotion voice cloning #1251

Open micelvrice opened 9 months ago

micelvrice commented 9 months ago

I use the method mentioned in thie rep https://github.com/innnky/emotional-vits to try to implement emotion voice cloning, I finetuned the pretrained synthsizer on a small dataset that contains about 24 speakers, each with 100 audio, and these 100 pieces of audio are divided by into roughly five or four categories, with the same tetx in each category but with different emotions. I inference with the finetuned synthesizer and the pretrained encoder and vocoder, but it's not working very well, if anyone know what the problem is or how it should be trained?

MassEast commented 8 months ago

I am not sure about the quality either. If I use the samples provided, I can generate reasonably good speech. If I use my own (e.g., by recording it through the UI), I was not able to produce any valuable output.