[Bug]: 2024-11-24 11:02:13.754948083 [E:onnxruntime:Default, provider_bridge_ort.cc:1992 TryGetProviderInfo_CUDA] /onnxruntime_src/onnxruntime/core/session/provider_bridge_ort.cc:1637 onnxruntime::Provider& onnxruntime::ProviderLibrary::Get() [ONNXRuntimeError] : 1 : FAIL : Failed to load library libonnxruntime_providers_cuda.so with error: libcublasLt.so.11: cannot open shared object file: No such file or directory

[abdelkareemkobo]

What happened?

when i run this code embedding_model = LateInteractionTextEmbedding("jinaai/jina-colbert-v2",cuda=True) I get 2024-11-24 11:02:13.754948083 [E:onnxruntime:Default, provider_bridge_ort.cc:1992 TryGetProviderInfo_CUDA] /onnxruntime_src/onnxruntime/core/session/provider_bridge_ort.cc:1637 onnxruntime::Provider& onnxruntime::ProviderLibrary::Get() [ONNXRuntimeError] : 1 : FAIL : Failed to load library libonnxruntime_providers_cuda.so with error: libcublasLt.so.11: cannot open shared object file: No such file or directory

2024-11-24 11:02:13.754961467 [W:onnxruntime:Default, onnxruntime_pybind_state.cc:965 CreateExecutionProviderInstance] Failed to create CUDAExecutionProvider. Require cuDNN 8. and CUDA 11.. Please install all dependencies as mentioned in the GPU requirements page (https://onnxruntime.ai/docs/execution-providers/CUDA-ExecutionProvider.html#requirements), make sure they're in the PATH, and that your GPU is supported. /home/kareem/mambaforge/envs/IR/lib/python3.11/site-packages/fastembed/common/onnx_model.py:89: RuntimeWarning: Attempt to set CUDAExecutionProvider failed. Current providers: ['CPUExecutionProvider'].If you are using CUDA 12.x, install onnxruntime-gpu via pip install onnxruntime-gpu --extra-index-url [https://aiinfra.pkgs.visualstudio.com/PublicPackages/_packaging/onnxruntime-cuda-12/pypi/simple/](https://aiinfra.pkgs.visualstudio.com/PublicPackages/_packaging/onnxruntime-cuda-12/pypi/simple/%60) warnings.warn(

What is the expected behaviour?

the model work with cuda version

A minimal reproducible example

No response

What Python version are you on? e.g. python --version

python3.11 on Pop!_OS ❯ nvcc --version nvcc: NVIDIA (R) Cuda compiler driver Copyright (c) 2005-2022 NVIDIA Corporation Built on Wed_Sep_21_10:33:58_PDT_2022 Cuda compilation tools, release 11.8, V11.8.89 Build cuda_11.8.r11.8/compiler.31833905_0

FastEmbed version

v0.4.2

What os are you seeing the problem on?

Linux

Relevant stack traces and/or logs

No response

[hh-space-invader]

Please verify your CUDA drivers are properly installed and compatible with cuDNN. Refer to these instructions.

[abdelkareemkobo]

@hh-space-invader Thanks man, I didn't notice that there was fastembed version for GPU, i was reading the qdrant documenation about fastembed. keep doing this amazing work. Nice to see you here :love_you_gesture: