search

Darwinkel / ShepardTTS

ShepardTTS is a fine-tuned XTTS v2.0.3 model, trained on paired dialogue/audio samples from the Mass Effect 2 and Mass Effect 3 base games.
https://shepardtts.darwinkel.net/
GNU General Public License v3.0
7 stars 0 forks source link

Add multilingual audio and text support #10

Open Darwinkel opened 8 months ago

Darwinkel commented 8 months ago

XTTS v2.0.3 supports English (en), Spanish (es), French (fr), German (de), Italian (it), Portuguese (pt), Polish (pl), Turkish (tr), Russian (ru), Dutch (nl), Czech (cs), Arabic (ar), Chinese (zh-cn), Japanese (ja), Hungarian (hu), Korean (ko), and Hindi (hi).

The Mass Effect games have dialogue and audio for English, French, German, Italian, Japanese, Spanish, Russian, and Polish. These are voiced by different actors. All of these languages are already supported by XTTS.

It should be reasonably easy to expand the training pipeline and modify the demo such that users can select a language. This will results in an exploding amount of available voices, though. Also, the training time will increase by ~8x.

Darwinkel commented 7 months ago

Partially implemented. Users can select a language which works surprisingly well. However, the model should still be finetuned on multilingual data.