search

Ki6an / fastT5

⚡ boost inference speed of T5 models by 5x & reduce the model size by 3x.
Apache License 2.0
565 stars 72 forks source link

Failed to create CUDAExecutionProvider #53

Closed ekaterinatretyak closed 2 years ago

ekaterinatretyak commented 2 years ago

Hi,

After having obtained ONNX models (not quantized), I would like to run inference on GPU devices with setting onnx runtime:

model_sessions = get_onnx_runtime_sessions(model_paths, default=False, provider=['CUDAExecutionProvider'])

However, I get the following error:

Failed to create CUDAExecutionProvider. Please reference https://onnxruntime.ai/docs/reference/execution-providers/CUDA-ExecutionProvider.html#requirements to ensure all dependencies are met.

I checked that all dependencies are installed. How could I fix it? Thanks in advance for answer

Ki6an commented 2 years ago

make sure to uninstall onnxruntime and install the right version of onnxruntime-gpu depending on the cuda and cudnn versions available on your device