Support Japanese voice cloing

[junwchina]

Hi, Thanks for your work to make voice cloning possible on Bark. I created datasets for Japanese and trained a Japanese quantizer. The result is pretty good after 24 epochs with around 5k data. If anyone wants to give it a try, they can simply download it from Huggingface.

japanese-quantizer Japanese datasets

[gitmylo]

Nice, for proper credit, you should create a pull request to https://github.com/gitmylo/Voice-cloning-quantizers, which has an index of voice cloning quantizers. Just add it to the models.json

[junwchina]

pr created.

[Maverick1983]

@junwchina after trained model, how can use for clone voice?

[junwchina]

@junwchina after trained model, how can use for clone voice?

You can use tts-generation-webui to clone and test voice. Remember to replace the qunatizer model to your trained model.

[Naozumi520]

@junwchina

I'm happy to see we can actually add a new language to bark. Can you please guide me to add an language to it (Cantonese)? I have few issue about this:

1. bark-data-gen generate training data yes. But for unsupported language we cannot use it directly. How did u overcome this issue and use your own dataset to train your Japanese quantizer?

2. How should the dataset looks like?

3. Other steps?

I'll be super grateful if you can help, thank you.

Update: My discord is .naozumi, it'll be easier for me to talk in there!

[junwchina]

bark-data-gen is a tool used to generate training data and train your quantizer model. The models we use to generate training data come from Bark itself, so It should support Chinese/Cantonese very well.

You need to follow these steps:

• Generate semantic data from text(Chinese/Cantonese content).
• Generate wav files from above semantic data.
• Train your Cantonese model from wavs and semantic data. Wav files is input of this model, semantic data is the output.

For more details, you can check my train script .

[junwchina]

I am going to close this issue. It's better to create new issue If you still have other problems.