Any plans of adding GPU support for linux?

sandrohanea / whisper.net

Whisper.net. Speech to text made simple using Whisper Models

MIT License

538 stars 83 forks source link

Any plans of adding GPU support for linux? #128

Closed PeterBagnegaard closed 10 months ago

PeterBagnegaard commented 10 months ago

I'm developing a feature that requires Whisper to be fairly responsive. It would be nice if I could use my GPU instead of CPU, but as far as I can tell this feature isn't implemented for anything but Windows. Are there any plans to change this?

sandrohanea commented 10 months ago

Indeed, the runtimes are currently only built for Windows with Cublas / OpenBlas Also, CoreML is used for Apple devices, but currently there is no whisper.net runtime on linux.

However, if someone builds it from the original whisper.cpp, it should work with the whisper.net which is just a wrapper around that.

Indeed, the preferred solution would be to have the runtime for linux as well, but unfortunately, I didn't have time for it yet.

sandrohanea commented 10 months ago

Cublas support for Linux - x64 is now added in Whisper.net.Runtime.Cublas 1.5.0

Clblast is not supported yet.