search

ClemensGruber / climart-gptree

MIT License
2 stars 1 forks source link

Research run Whisper as ASR on-device #4

Closed ClemensGruber closed 8 months ago

ClemensGruber commented 8 months ago

Running Whisper locally instead of as a service could make transcription faster, strengthen privacy and reduce costs. So let's investigate how accurate and fast this is possible on a Raspberry Pi!

ClemensGruber commented 8 months ago

We could run OpenAI Whisper with the models tiny, baseand small on a Raspberry Pi 5. But accuracy is - especially for German speech - not sufficient. ASR with OpenAI Whisper via service / API and so the use of a better language model is more accurate and also fast without the need of faster hardware (RasPi 4 vs. 5), So we'll stick with the API solution for now, even if it causes costs.

For a tutorial how to install Whisper on-device on a RasPi 5 and an English and German transcription test with C++ on device vs. API see https://community.hiveeyes.org/t/openai-whisper-auf-einem-raspberrypi-5-on-device/5174