MotazSabri / Hanami-release

Live translator that captures any audio that comes from a WINDOWS speaker or microphone and translates it to the desired language.
33 stars 7 forks source link

Better Translation? #1

Open codenan42 opened 1 year ago

codenan42 commented 1 year ago

What an amazing software. It works just fine from the minute I install it and run it. Though the translation accuracy is pretty off most of the time (probably due to the transcription?) and that is just me using it for Japanese > English translation.

Any plans on using it with openai whisper? from my experience it has better results. Although I'm not sure if its capable of doing fast transcribing + translation with the speed that you have now on your software.

Either way great job. I've been looking for alternative because I spent so much time using openai + whisper to transcribe and translate japanese video for just my entertainment. Having an alternative like this is very useful. Also it runs and work so smoothly which is a plus.

MotazSabri commented 1 year ago

Thanks for your neat remark. As you kindly mentioned, the translation is far from perfect. I tried whisper integration but the app become slow (I7 machine without GPU) and the covered languages were not many.

There are other fronts I can tackle to improve the quality.

I know users will always prefer high performance and high quality translation at the same time. I will stick to that as much as I can.

Thanks again for the points.

MidnightKittenCat commented 8 months ago

Thanks for your neat remark. As you kindly mentioned, the translation is far from perfect. I tried whisper integration but the app become slow (I7 machine without GPU) and the covered languages were not many.

There are other fronts I can tackle to improve the quality.

  • Improve the speech to text through AI.
  • Normalize text before sending it to the translation. Also processing it after translation.
  • Make Whisper usage an optional feature (through a tick) for users with robust machines maybe with GPU.

I know users will always prefer high performance and high quality translation at the same time. I will stick to that as much as I can.

Thanks again for the points.

Any news on this happening?

MotazSabri commented 8 months ago

Any news on this happening?

Thanks for your follow-up.

I've released two updates since your message was posted, but unfortunately, I haven't achieved a significant improvement in the translation quality yet. I'm grateful for your continued use of the application, and I remain hopeful that it will continue to evolve.

MidnightKittenCat commented 8 months ago

Any news on this happening?

Thanks for your follow-up.

I've released two updates since your message was posted, but unfortunately, I haven't achieved a significant improvement in the translation quality yet. I'm grateful for your continued use of the application, and I remain hopeful that it will continue to evolve.

Will there ever be support to have it locally hosted? which would allow gpu acceleration

MotazSabri commented 8 months ago

Will there ever be support to have it locally hosted? which would allow GPU acceleration

With the absence small language model that can do translation and speech-to-test, I think cloud hosting is the most reliable option. As mentioned on the main page, we use Deepl and Google Translate for this for now.