myshell-ai / DreamVoice

69 stars 7 forks source link

Plugin mode for other VC model? #9

Open ShawnPi233 opened 1 week ago

ShawnPi233 commented 1 week ago

Thanks for your great work! But I just don't understand how can I use the DreamVG model in other zero-shot VC models, since the speaker embedding predicted from the text prompt can't share the same embedding space with other VC models. Should we finetune the VC model with the speaker embedding from DreamVG?

haidog-yaqub commented 16 hours ago

hi there, actually you can train a dreamvc plugin on the speaker embedding space used in the pretrained vc model by keep vc model frozen. As the dreamvc plugin is super light, it can be trained in just few hours on the dreamvc db.