I have cuda installed and new version of buzz, still it uses CPU rather than GPU

[Walkerby]

My gpu is 4070ti. Cuda version is 12.5.1 on windows. Buzz version is 1.0.1.

[raivisdejus]

Whisper.cpp will use CPU, that is expected.

Can you test all of these:

• Whisper
• Faster Whisper
• Huggingface for example with openai/whisper-small

Each of them have a different mechanism to enable CUDA, so curious if all of them do not work.

Also what CUDA version do you have?

Will try to figure this out...

[Walkerby]

Whisper.cpp will use CPU, that is expected.

Can you test all of these:

• Whisper
• Faster Whisper
• Huggingface for example with openai/whisper-small

Each of them have a different mechanism to enable CUDA, so curious if all of them do not work.

Also what CUDA version do you have?

Will try to figure this out...

I tried whisper and faster whisper model. Both of them can only run on CPU. My cuda version is cuda_12.5.1_555.85_windows.

[raivisdejus]

@Walkerby Please see this https://github.com/chidiwilliams/buzz/discussions/861 It has some notes on how you may be able to make it work while we figure out the official solution.

If you gt it working, please share some notes that may be helpful to others

[raivisdejus]

I was able to get CUDA GPU support working for Whisper and Huggingface whisper type. Tested on Windows server 2022 with CUDA 12.4

Install ffmpeg choco install ffmpeg

pip install ffmpeg
pip install ffmpeg-python

Install torch with CUDA support https://pytorch.org/get-started/locally/ f.e. pip3 install torch torchvision torchaudio --index-url https://download.pytorch.org/whl/cu124

Install appropriate buzz wheel from https://github.com/chidiwilliams/buzz/actions/runs/10221561439 f.e. pip install buzz_captions-1.0.2-cp312-cp312-win_amd64.whl

Run buzz python -m buzz and use some Huggingface model f.e. openai/whisper-large-v3

[raivisdejus]

See notes on GPU support on Windows here https://github.com/chidiwilliams/buzz/blob/main/CONTRIBUTING.md#gpu-support

And for best results use the latest development version from https://github.com/chidiwilliams/buzz/actions/workflows/ci.yml?query=branch%3Amain (to download log into the github)

Will close the issue, reopen if there are additional details

[LakeishaKowalczyk]

@raivisdejus Hello, followed the notes on GPU support on Windows, but no matter what I do, I just can’t seem to succeed.

Windows11 21H2 GPU RTX 2070S Driver 32.0.15.6081

Installed cuda_12.1.0_531.14_windows.exe

pip install buzz-captions
Successfully installed buzz-captions-1.1.0

pip3 install torch torchvision torchaudio --index-url https://download.pytorch.org/whl/cu121
Successfully installed torch-2.4.1+cu121 torchaudio-2.4.1+cu121 torchvision-0.19.1+cu121

pip3 install ffmpeg
Successfully installed ffmpeg-1.4

pip3 install ffmpeg-python
Successfully installed ffmpeg-python-0.2.0

python -m buzz
(with Faster Whisper(Large-v3))

Always use the CPU and never the GPU

[raivisdejus]

@LakeishaKowalczyk Let's debug this. Does Hugging Face Whisper or regular Whisper use GPU?

[LakeishaKowalczyk]

@LakeishaKowalczyk Let's debug this. Does Hugging Face Whisper or regular Whisper use GPU?

Tested with regular Whisper, using CPU only