GPU fails the run on the containerized version of the worker

greenw0lf commented 3 months ago

Stacktrace of the error:

Traceback (most recent call last):
  File "/src/worker.py", line 136, in <module>
    processing_result, full_provenance_chain = main_data_processor.run(
  File "/src/main_data_processor.py", line 85, in run
    proc_result = run_whisper(model_input)
  File "/src/whisper.py", line 43, in run_whisper
    model = faster_whisper.WhisperModel(
  File "/src/.venv/lib/python3.10/site-packages/faster_whisper/transcribe.py", line 133, in __init__
    self.model = ctranslate2.models.Whisper(
RuntimeError: CUDA failed with error named symbol not found

greenw0lf commented 3 months ago

Managed to get it working, I will document the steps taken in the wiki

cliffwoolley commented 3 months ago

If you were using driver 555.85 here, please see https://github.com/NVIDIA/nvidia-container-toolkit/issues/520 .

beeldengeluid / dane-whisper-asr-worker

GPU fails the run on the containerized version of the worker #33