scosman / voicebox

Exploration: using technology to aid people who lack both the ability to speak and fine motor control.
12 stars 0 forks source link

Whisper Model moved to CoreML #9

Open scosman opened 1 year ago

scosman commented 1 year ago

CPU usage is quite high on current whisper model. It's functional for evaluation, but the battery usage isn't shipable. Moving the model to CoreML would help a ton, offloading work from CPU and improving efficiency. Accelerate framework is using GPU/ATX/SIMD but not ANE.

This project has some work, but doesn't appear complete: https://github.com/vade/OpenAI-Whisper-CoreML

scosman commented 1 year ago

Perfect, the whispercpp folks are working on this! Branch to integrate when ready: https://github.com/ggerganov/whisper.cpp/pull/566