CPU usage is quite high on current whisper model. It's functional for evaluation, but the battery usage isn't shipable. Moving the model to CoreML would help a ton, offloading work from CPU and improving efficiency. Accelerate framework is using GPU/ATX/SIMD but not ANE.
CPU usage is quite high on current whisper model. It's functional for evaluation, but the battery usage isn't shipable. Moving the model to CoreML would help a ton, offloading work from CPU and improving efficiency. Accelerate framework is using GPU/ATX/SIMD but not ANE.
This project has some work, but doesn't appear complete: https://github.com/vade/OpenAI-Whisper-CoreML