search

niedev / RTranslator

Open source real-time translation app for Android that runs locally
Apache License 2.0
4.95k stars 367 forks source link

Support hebrew language #13

Open thewh1teagle opened 1 week ago

thewh1teagle commented 1 week ago

whisper.cpp supports Hebrew. Could you clarify the limitations that prevent rtranslator from supporting it? I noticed that ctranslate inference with NLLB also supports Hebrew.

This is a very innovative project! Thanks!

niedev commented 1 week ago

Thank you! I excluded all the languages with a WER higher than 36%, Hebrew has a WER of 44% with Whisper small, witch is practically unusable, in the future, however, I will try to expand the supported languages more and more (perhaps by integrating better or larger models, or by inserting an option in the settings to also support languages with lower quality, ecc.).

thewh1teagle commented 1 week ago

I would enable them even with the lower quality or at least provide option to enable them in the settings. I understand that in terms of translation the model works good enough with Hebrew? You can also provide option to click deep links which will download larger model. just like I did in the project vibe Relatively easy to implement and very effective to use deep links

niedev commented 1 week ago

Yes, in fact, adding the option to support low quality languages wouldn't be bad, I'll add it in the future (probably soon, but it depends on how much free time I have), as for the quality of the translation, I don't know (I only use as a criterion the quality of Whisper) since Meta in his paper does not give scores for the smaller NLLB models.

For the support of custom models, in my case it would not be easy, given that at the moment the models used (as you can see from the .onnx files of the 2.0.0 release) have been manually modified by me (to optimize performance and RAM consumption), so the inference logic of the the app only works for models modified by me in the same way. However in the future I plan to increase the models to choose from.

By the way, congrats for your project, it seems really cool!