jech / galene-stt

Speech-to-text support for Galene
https://galene.org
MIT License
3 stars 1 forks source link

Japanese is posted as English translated text #1

Closed matoken closed 3 months ago

matoken commented 3 months ago

Hello. I tried this project because it is interesting. I am a Japanese speaker, so I changed the model to base.en to base and tried it in Japanese.

and text is translated into English and posted to Galene. I want the option to make this translation feature off.

jech commented 3 months ago

Confirmed, this also happens for other languages.

Galene-stt is currently running the model with default options, and the default options are not very useful. We need to do two things:

Unfortunately, I couldn't find any good documentation for whisper-cpp (which is used by galene-stt), and I'm not sure whether the documentation for the Whisper API applies to whisper-cpp. The best source I have found is the whisper-cpp example: https://github.com/ggerganov/whisper.cpp/blob/master/examples/main/main.cpp#L1089

Your help would of course be warmly appreciated.

jech commented 3 months ago

I've just added the options -lang and -translate to control the behaviour. Please test with galene-stt -lang ja.

matoken commented 3 months ago

Hello. I tried the -lang ja option. This allowed Japanese to be output as Japanese! Thank you.

(Also, the accuracy of the base model for Japanese is poor, so I would like to use a larger model, but it seems that my CPU performance is insufficient. I need a new machine...)

jech commented 3 months ago

I would like to use a larger model, but it seems that my CPU performance is insufficient. I

Same here. However, I don't think a faster machine will help much, since whisper.cpp is currently unable to use more than 4 cores effectively. I suppose that the only solution would be to run it on the GPU, but I haven't had time to try it yet. See here for instructions: https://github.com/ggerganov/whisper.cpp.

Please report on the mailing list if you're successful.