search

sandrohanea / whisper.net

Whisper.net. Speech to text made simple using Whisper Models
MIT License
505 stars 77 forks source link

Updating to Whisper.cpp v1.6.2+ #200

Open abbotadmin opened 1 week ago

abbotadmin commented 1 week ago

Whisper .NET seems to be quite far behind the main releases of Whisper.cpp, including some major improvements to how it works with GPUs, etc.

I am curious if this framework is abandoned or whether this work is currently in progress. @sandrohanea would you mind providing some guidance here? Are you currently still maintaining the framework?

dfengpo commented 6 days ago

I have same question, looking forward to updates

sandrohanea commented 6 days ago

Hello all, It's not abandoned, but indeed, I didn't have that much time to respond to all the issues that people encountered lately (mostly loading different runtimes and sometimes not supported ones).

Regarding the upgrade, it is due indeed and will try to do it soon.

abbotadmin commented 6 days ago

That's great news. Of course you can only do what you have time for and I'm currently using what you've got here to great effect so thanks for the hard work so far.

Do you think the whispercpp upgrade itself is a lot of work? It's something I was considering looking into myself but alas I too am short on time at the moment.

sandrohanea commented 6 days ago

WIP branch is here: https://github.com/sandrohanea/whisper.net/pull/201 in case someone wants to play with it and provide some early feedback :)

I managed to make it work with Windows x64, but for the other platforms / GPUs, there is still some work to be done.

dfengpo commented 5 days ago

Can you give me a rough idea of the implementation plan? I am willing to try to make some contributions

Sing303 commented 5 days ago

Will it be possible to do https://github.com/sandrohanea/whisper.net/issues/114 within this PR ? Very important feature (if you have time)

sandrohanea commented 5 days ago

Hello @Sing303, It seems that Clblast was removed from whisper.cpp: https://github.com/ggerganov/whisper.cpp/commit/3b1ac0382892fa74a1b47d9b2e07ab4917d4cd1e

However, Cuda will still be available. To switch from Cuda (GPU) to CPU, Ill try to see if useGpu: can be used with the newer version, but initially, I just try to upgrade it and create some preview version.

Sing303 commented 5 days ago

I think a CUDA backend with Gpu = false won't work if there is no Nvidia video card anyway. But we need to check

abbotadmin commented 4 days ago

@sandrohanea I thought I'd give the WIP 1.6.0 a go however unfortunately it looks like your LFS quota is maxed out and the clone doesn't complete sadly.