search

ahmetoner / whisper-asr-webservice

OpenAI Whisper ASR Webservice API
https://ahmetoner.github.io/whisper-asr-webservice
MIT License
1.99k stars 358 forks source link

Intel ARC / XPU Support for Whisper ASR – An Open Invitation #148

Open Sikerdebaard opened 11 months ago

Sikerdebaard commented 11 months ago

Hello Whisper ASR Team,

I've been having a blast using your web service. Inspired by your work, I decided to roll up my sleeves and add Intel ARC / XPU support. It's been quite the adventure, especially with the twists and turns of the Intel ARC ecosystem for deep learning.

Here's the catch though – the Docker image turned out to be a bit on the larger side, hitting the scales at about 25 GiB. So, I've held off on a pull request for now because that's quite a load to consider!

I'm reaching out to see if you're interested in this Intel ARC / XPU compatibility. If you are, my repo is ready and waiting to help as a starting point for weaving in support on your end. No pressure, though. Just thought I'd share what I've been up to and see if it could contribute to your toolkit.

If this sounds like something you'd want to explore, or if you have any insights, I'm here and happy to chat more.

My repo: https://github.com/Sikerdebaard/whisper-asr-webservice-xpu

ayancey commented 11 months ago

Please give us some time to review this. I don't have an ARC card, so testing could be difficult. Do you, @ahmetoner ?

Oyo95 commented 6 months ago

Hi !

any news about the compatibility with Intel gpu arc ? :)

ChezRD commented 4 months ago

As far as I know, ctranslate2 has no support for XPU, so no faster-whisper too. I used leuc/whisper on my ARC 750 with 8GB VRAM. This amount of VRAM can handle only medium model.

bonswouar commented 3 months ago

As far as I understand XPU are supported by huggingface transformers though, so it could be supported with #146

Awlexus commented 2 months ago

I've tried to get it running, but encountered a couple of problems on the way

  1. Error with intel repository 
    3.247 W: GPG error: https://repositories.intel.com/gpu/ubuntu jammy InRelease: The following signatures couldn't be verified because the public key is not available: NO_PUBKEY 28DA432DAAC8BAEA
3.247 E: The repository 'https://repositories.intel.com/gpu/ubuntu jammy InRelease' is not signed.

    I solved this by adding the following snippet taken from https://dgpu-docs.intel.com/driver/installation.html

    RUN sudo apt-get install -y gpg-agent wget && \
    wget -qO - https://repositories.intel.com/gpu/intel-graphics.key | \
sudo gpg --yes --dearmor --output /usr/share/keyrings/intel-graphics.gpg && \
    echo "deb [arch=amd64 signed-by=/usr/share/keyrings/intel-graphics.gpg] https://repositories.intel.com/gpu/ubuntu jammy/lts/2350 unified" | \
sudo tee /etc/apt/sources.list.d/intel-gpu-jammy.list
  2. Model does not seem to be using GPU This line here seems to tell me that it's using the CPU instead of the GPU. Looking at the CPU usage with top and the duration for transcribing anything confirms that theory. I have not found out how to fix that. 
    /usr/local/lib/python3.10/dist-packages/whisper/transcribe.py:117: UserWarning: FP16 is not supported on CPU; using FP32 instead