search

t41372 / Open-LLM-VTuber

Talk to LLM by voice with Live2D that runs offline on multiple platforms. An attempt to build AI VTuber neuro-sama.
MIT License
39 stars 3 forks source link

Can you support whisper.cpp TTS #4

Open DDXDB opened 2 weeks ago

DDXDB commented 2 weeks ago

whisper.cpp https://github.com/ggerganov/whisper.cpp

Support for OpenVino acceleration would be even better!

t41372 commented 1 week ago

I have added support for Whisper.cpp with the Python binding pywhispercpp. However, I don't think the current implementation support OpenVino. It doesn't even support GPU acceleration on Mac.

I'm having trouble getting all the acceleration working with the Python binding, so I'll probably leave the Python binding as it is and instead run Whisper.cpp as a server and use the API to interact with it. The user can compile the Whisper.cpp that works best on their system.

t41372 commented 1 week ago

I get it to work with mac GPU acceleration, and I probably won't be doing the server thing. I'll take a look at the OpenVino in the future.