search

microsoft / onnxruntime

ONNX Runtime: cross-platform, high performance ML inferencing and training accelerator
https://onnxruntime.ai
MIT License
14.73k stars 2.94k forks source link

Please clarify the default CUDA support of onnxruntime 1.17.0 #19356

Closed henryruhs closed 9 months ago

henryruhs commented 9 months ago

Describe the issue

I do not fully understand if onnxruntime-gpu 1.17.0 supports both CUDA 11.8 and CUDA 12.1 or if CUDA 12.1 needs a certain --extra-index-url to a stable channel.

If that is the case, why is this not easy to find in your docs?

To reproduce

I used --extra-index-url https://pkgs.dev.azure.com/onnxruntime/onnxruntime/_packaging/onnxruntime-cuda-12/pypi/simple for CUDA 12.1 support but it does not seem stable and kinda broken some of my fp16 model inferences (black boxes).

Urgency

No response

Platform

Linux

OS Version

Ubuntu 22

ONNX Runtime Installation

Released Package

ONNX Runtime Version or Commit ID

1.17.0

ONNX Runtime API

Python

Architecture

X64

Execution Provider

CUDA

Execution Provider Library Version

CUDA 12.1

tianleiwu commented 9 months ago

https://onnxruntime.ai/docs/install/#python-installs

The dependent packages need to be installed from pypi first, then install onnxruntime-gpu from azure feeds:

pip install coloredlogs flatbuffers numpy packaging protobuf sympy
pip install onnxruntime-gpu --index-url https://pkgs.dev.azure.com/onnxruntime/onnxruntime/_packaging/onnxruntime-cuda-12/pypi/simple --no-cache-dir
henryruhs commented 9 months ago

Thanks for the clarification. So I made it correct already.