Error When Running inference.py with CUDA 12.1 on Server

[hunglsxx]

Hello, I followed the latest instructions when installing on a server using CUDA 12.1. However, when I run python inference.py, I encounter an error as shown in the image. Please help. I'm using Ubuntu 20

[zzzweakman]

Hi @hunglsxx,

You can try the following commands:

pip uninstall onnxruntime-gpu
pip install onnxruntime-gpu --extra-index-url https://aiinfra.pkgs.visualstudio.com/PublicPackages/_packaging/onnxruntime-cuda-12/pypi/simple/

---

[hunglsxx]

Hi @hunglsxx,

You can try the following commands:

pip uninstall onnxruntime-gpu
pip install onnxruntime-gpu --extra-index-url https://aiinfra.pkgs.visualstudio.com/PublicPackages/_packaging/onnxruntime-cuda-12/pypi/simple/

@zzzweakman Thanks for your reply, but it's still not working.

[liuxuyao]

你好@hunglsxx，

您可以尝试以下命令：

pip uninstall onnxruntime-gpu
pip install onnxruntime-gpu --extra-index-url https://aiinfra.pkgs.visualstudio.com/PublicPackages/_packaging/onnxruntime-cuda-12/pypi/simple/

good

[Misty1113]

The reason for this issue is that the versions of onnxruntime, CUDA, and CUDNN are incompatible. The solution is as follows: uninstall current onnxruntime, then check the appropriate onnxruntime version at the official link and reinstall it: https://onnxruntime.ai/docs/execution-providers/CUDA-ExecutionProvider.html#requirements

The instruction example is as follows:

pip uninstall onnxruntime-gpu
pip install onnxruntime==1.18.0