search

erew123 / alltalk_tts

AllTalk is based on the Coqui TTS engine, similar to the Coqui_tts extension for Text generation webUI, however supports a variety of advanced features, such as a settings page, low VRAM support, DeepSpeed, narrator, model finetuning, custom models, wav file maintenance. It can also be used with 3rd Party software via JSON calls.
GNU Affero General Public License v3.0
816 stars 91 forks source link

Non-english character support in TTS Generator #188

Closed ytt246 closed 4 months ago

ytt246 commented 4 months ago

Currently TTS Generator doesn't seem to work with all languages supported by XTTS V2. For example, using arabic, chinese, and japanese characters results in an audio clip of only a second long of incoherent mumbling. Not sure what could've caused this, but it works fine in text-gen-webui

erew123 commented 4 months ago

Hi @ytt246

Ive made a small update the the TTS generator you can git pull and see if that changes anything.

Personally I do not speak or read arabic, chinese, and japanese, so if you are able to do so and still have an issue, Perhaps you would send me short examples of the text you are trying to generate within the TTS generator then output from the command prompt/terminal so that I can understand what it is or isnt passing over.

Thanks

ytt246 commented 4 months ago

The problem still exists. I've attached the command prompt output screenshot with three tests:

  1. this is a short test using english
  2. 这是一个使用中文的测试
  3. これは日本語を使ったテストです

tts ch-jp-test-audio.zip

It has the same behaviour with Arabic.

In the case of Korean (이건 한국어로 된 시험이에요), nothing happens after clicking "Generate TTS", no new lines show up and no audio file is generated.

In the case of Hindi (यह हिंदी में एक परीक्षा है), the characters are sent to the command prompt but no audio file is generated.

hindi

erew123 commented 4 months ago

Hi @ytt246

Hindi Hindi is undocumented by Coqui and is only supported on the 2.0.3 model. Please see my notes on this here regarding setting up the 2,0,3 model on AllTalk 1.9 and also the reference from the main github page.

image

With the other languages, I have changed a few bits. If you want to git pull again and give it a go.

image

Thanks

ytt246 commented 4 months ago

I'm glad to report it all works now, thanks!