Train New Voice Conversion Model

HRNPH / AIwaifu

Open-Waifu open-sourced finetunable customizable simpable AI waifu inspired by neuro-sama

GNU General Public License v2.0

402 stars 31 forks source link

Train New Voice Conversion Model #28

Open HRNPH opened 1 year ago

HRNPH commented 1 year ago

basically current model of Voice Conversion(SoVits) pushed on #26 #27 are SHIT due to a low amount of training currently stored in Huggingface https://huggingface.co/openwaifu/SoVits-VC-Chtholly-Nota-Seniorious-0.1 which made it sound shitter than our old pipeline what we need to do is just train it more and maybe add more VC model Todo:

Train https://huggingface.co/openwaifu/SoVits-VC-Chtholly-Nota-Seniorious-0.1
Add new VC model if you wanna help maybe try training your own model using my uncleaned training notebook right here and adding a list of model from hugginface take a look at our voice conversion pipeline here

andri-jpg commented 1 year ago

Instead of using TTS and Sovits together, why don't we just use TTS Vits?

HRNPH commented 1 year ago

The reason is when you want to train your own custom Voice TTS Model is hard to train Voice Conversion is easier to train since you only need an audio sample and not an audio-text pair

andri-jpg commented 1 year ago

okay, I understand, right now I'm trying to retrain the model. Maybe I'll try with a pretrained model, then I'll fine tune with the existing dataset.

HRNPH commented 1 year ago

I have pre-trained model and dataset if you need one

andri-jpg commented 1 year ago

I have pre-trained model and dataset if you need one

OK, I'll give it a try, can you send me the link?

HRNPH commented 1 year ago

This for datasets: https://drive.google.com/file/d/102PVFeKrYu8Pfo-4NBC5jsgVbHNiP_Uc/view?usp=share_link This for SoVits model: https://drive.google.com/drive/folders/1-5YkGLEmLvfFEE1x5cG08KdDrrIWf8pL?usp=share_link (this is the same model that was used as current base model)