search

BramNH / wyoming-faster-whisper-docker-cuda

1 stars 0 forks source link

CTranslate2 CUDA Support #1

Open FyrbyAdditive opened 1 month ago

FyrbyAdditive commented 1 month ago

Tried this out today and it fails on my (ARM) AGX Orin on 22.04/Jetpack 6 due to CTranslate2 missing CUDA support. Docker logs:

Traceback (most recent call last):
  File "/usr/lib/python3.10/runpy.py", line 196, in _run_module_as_main
    return _run_code(code, main_globals, None,
  File "/usr/lib/python3.10/runpy.py", line 86, in _run_code
    exec(code, run_globals)
  File "/usr/local/lib/python3.10/dist-packages/wyoming_faster_whisper/__main__.py", line 149, in <module>
    run()
  File "/usr/local/lib/python3.10/dist-packages/wyoming_faster_whisper/__main__.py", line 144, in run
    asyncio.run(main())
  File "/usr/lib/python3.10/asyncio/runners.py", line 44, in run
    return loop.run_until_complete(main)
  File "/usr/lib/python3.10/asyncio/base_events.py", line 649, in run_until_complete
    return future.result()
  File "/usr/local/lib/python3.10/dist-packages/wyoming_faster_whisper/__main__.py", line 119, in main
    whisper_model = faster_whisper.WhisperModel(
  File "/usr/local/lib/python3.10/dist-packages/faster_whisper/transcribe.py", line 133, in __init__
    self.model = ctranslate2.models.Whisper(
ValueError: This CTranslate2 package was not compiled with CUDA support

I tried the following in case it makes a difference with the same result.

BramNH commented 1 month ago

The image build completes without issue? I am not familiar with the Nvidia Jetson products, but I am not sure if you can directly use the cudnn8-runtime-ubuntu images images. However Jetpack 6 seems to support CUDA 12.2 and cudnn8 8.9....

You might need to add CMake argument: -DWITH_CUDA=ON to compile it with CUDA support, as mentioned here.