search

MichaelMcCulloch / WhisperVoiceKeyboard

A voice to text keyboard based on OpenAI Whisper Model.
MIT License
47 stars 5 forks source link

Consider switching to whisper.cpp #10

Open djmaze opened 1 year ago

djmaze commented 1 year ago

Whisper.cpp is a high-performance alternative to the original implementation. There is also an Android example included. Might make sense to switch to it?

MichaelMcCulloch commented 1 year ago

Amazed I didnt see this before. Thank you.

djmaze commented 1 year ago

Mhh.. Does whisper.cpp really deprecate this? AFAICS there is no voice keyboard application using whisper.cpp yet, and this would be really nice to have.

MichaelMcCulloch commented 1 year ago

Hey that's a good point. Sorry, that was a knee-jerk reaction. I've only just cloned whisper.cpp and I'm looking into integrating it.

MichaelMcCulloch commented 1 year ago

The tiny and base models cant understand my friend say 'Burt Reynolds'. The medium comes close, but it takes a minute and a half, instead of seconds. I would want to use the medium or, ideally large models on the GPU. But the repo intentionally doesn't support GPU, however there's an entry point mentioned in the FAQ.

MichaelMcCulloch commented 1 year ago

Given that whisper.cpp intentionally does not support GPU and was built for CPU with a 'bespoke' 'tensorflow' implementation, it would require more effort on my part to get it working with the GPU, than to modify the existing model to work with the GPU or NNAPI.

Unless someone with expertise in android GPU or NNAPI hardware would be willing to pair on this solution path, I'm going to mark this as wontfix