Hope the "onnxruntime" issue is resolved soon！

[chaorenai]

I have been tortured by onnxruntime for a long time recently. The author said it is because CUDA must be 11.8 before it can be used. The current comfyui official version is 12.1, and my system is 12.3. Downgrading to 11.8 is a huge problem. Later, I discovered that there are many plug-ins that use onnxruntime. All the .onnx models, such as was, such as comfyui-reactor-node; I only need to install plug-ins such as comfyui-reactor-node and was in advance, and then install cn, and the problem will be solved. Perfectly solved. If you install controlnet_aux/ first, controlnet_aux/ will mess up everything. This is not my problem alone, many friends are encountering similar problems now!

Since plug-ins such as comfyui-reactor-node can solve this problem, I believe controlnet_aux/ can also do it.

[chaorenai]

There is currently no need to downgrade to cuda11.8, other plug-ins can use onnxruntime normally.

[Fannovel16]

@yinghanzi Are you using Windows?

[murphylanga]

I have the same problem and use Windows

[f-rank]

Getting this on a new comfyui portable install, on windows.

[Layer-norm]

Maybe you can disable you GPU acceleration for onnx as reactor-node suggestted https://github.com/Gourieff/comfyui-reactor-node/discussions/73

[Fannovel16]

@Layer-norm Opencv already has built-in onnx loader which uses CPU by default. Is onnxruntime faster than that?

[Layer-norm]

@Fannovel16 Yes, gpu is much more faster, especially for video. For a 720p video around 300 frames, onnxruntime-gpu only takes about 1 minute to finsh the task, while cpu takes around 1hours. And if you talk about onnxruntime in cpu, I can't tell the difference.

[Layer-norm]

I am sorry that onnxruntime-gpu may not support cuda12.3 now. If you wan't to use onnxruntime-gpu in cuda12.x and above in windows, you can try to build it yourself instead of using prebuilt wheel, This link may help you https://onnxruntime.ai/docs/build/eps. And you can find scripts source from here. However, it only support cuda version below 12.2 now.

[tianleiwu]

To use CUDA 12.* instead of 11.8, you can try install nightly binary like the following:

pip uninstall onnxruntime onnxruntime-gpu ort-nightly-gpu ort-nightly
python -m pip install coloredlogs flatbuffers numpy packaging protobuf sympy
pip install ort-nightly-gpu --index-url=https://aiinfra.pkgs.visualstudio.com/PublicPackages/_packaging/ort-cuda-12-nightly/pypi/simple/

[Layer-norm]

@f-rank @tianleiwu @murphylanga onnxrutime-gpu support cuda12 now. YOU SHOULD REMOVE THE OLD VERSION COMPLETELY pip uninstall onnxruntime onnxruntime-gpu and remove onnxruntime folder in (your python folder)/Lib/site-packages if it exists after uninstall. Then pip install -U onnx onnxruntime onnxruntime-gpu after that you can use onnx model with your gpu in cuda12 now. 请先完全卸载onnxruntime onnxruntime-gpu 并在卸载后确认python embeded/Lib/site-packages文件夹里是否有onnxruntime文件夹残留 如果有请将整个onnxruntime文件夹扔进回收站（命令可能卸载不干净） 然后重新安装onnx onnxruntine onnxruntime-gpu python -m pip install -U onnx onnxruntime onnxruntime-gpu

[murphylanga]

👍 Thank you so much for your assistance in solving the problem.

[escalate007]

@f-rank @tianleiwu @murphylanga onnxrutime-gpu support cuda12 now. YOU SHOULD REMOVE THE OLD VERSION COMPLETELY pip uninstall onnxruntime onnxruntime-gpu and remove onnxruntime folder in (your python folder)/Lib/site-packages if it exists after uninstall. Then pip install -U onnx onnxruntime onnxruntime-gpu after that you can use onnx model with your gpu in cuda12 now. 请先完全卸载onnxruntime onnxruntime-gpu 并在卸载后确认python embeded/Lib/site-packages文件夹里是否有onnxruntime文件夹残留 如果有请将整个onnxruntime文件夹扔进回收站（命令可能卸载不干净） 然后重新安装onnx onnxruntine onnxruntime-gpu python -m pip install -U onnx onnxruntime onnxruntime-gpu

非常感谢，困扰我3个小时的问题，各种找解决方案，这个最有效。 我的controlnet插件没出问题，是rembg移除背景的插件报错。

[tianleiwu]

python -m pip install -U onnx onnxruntime onnxruntime-gpu is NOT the right way. It installs ORT for CUDA 11, not for CUDA 12. Also, there is no need to install onnxruntime package (it is CPU only), which might overwrite the gpu package.

Please make sure you only install one ORT package (like onnxruntime, onnxruntime-gpu or nightly variants). It is because all those packages will be installed to same onnxruntime subdirectory under site-packages (run pip show onnxruntime-gpu to see the site-packages location).

Sometime, the issue is caused by some dependency (like VC Runtime, CUDA and cuDNN) not installed. Please follow https://onnxruntime.ai/docs/install/ to install all those requirements for onnxruntime-gpu for CUDA 12. If you installed latest pytorch for cuda 12 for comfyui, you can also place import torch before import onnxruntime to reuse those DLLs that loaded by torch.

The command to install official package is like the following:

pip install onnxruntime-gpu --extra-index-url https://aiinfra.pkgs.visualstudio.com/PublicPackages/_packaging/onnxruntime-cuda-12/pypi/simple/

[shivshankar11]

for late comer: here's the way to enable gpu accelerate on cuda12.x

track the issue here for version changes: https://github.com/microsoft/onnxruntime/issues/13932 runtime nightly: https://dev.azure.com/onnxruntime/onnxruntime/_artifacts/feed/onnxruntime-cuda-12 ort nightly:https://aiinfra.visualstudio.com/PublicPackages/_artifacts/feed/ort-cuda-12-nightly

with cu12.*

pip install coloredlogs flatbuffers numpy packaging protobuf sympy pip install ort-nightly-gpu --index-url=https://aiinfra.pkgs.visualstudio.com/PublicPackages/_packaging/ort-cuda-12-nightly/pypi/simple/ pip install onnxruntime-gpu==1.17.0 --index-url=https://pkgs.dev.azure.com/onnxruntime/onnxruntime/_packaging/onnxruntime-cuda-12/pypi/simple/