search

microsoft / DirectML

DirectML is a high-performance, hardware-accelerated DirectX 12 library for machine learning. DirectML provides GPU acceleration for common machine learning tasks across a broad range of supported hardware and drivers, including all DirectX 12-capable GPUs from vendors such as AMD, Intel, NVIDIA, and Qualcomm.
MIT License
2.15k stars 286 forks source link

Onnxruntime compiling stuck , cannot exit. #607

Open Jay19751103 opened 1 month ago

Jay19751103 commented 1 month ago

Describe the issue When use test data to do ort.InferenceSession, takes time and not exit

To reproduce use the onnx file put in google drive https://drive.google.com/file/d/1y-evMcenYe-Q0JpyQvuwO0YHBX8gSSBE/view?usp=sharing import numpy as np import onnxruntime as ort

EP_list = ['DmlExecutionProvider'] sess_opt = ort.SessionOptions() sess_opt.log_severity_level = 0 sess_opt.graph_optimization_level = ort.GraphOptimizationLevel.ORT_ENABLE_BASIC sess = ort.InferenceSession("text_encoder_3.onnx", sess_opt, providers=EP_list)

use win debug see that the system is doing TryCreateCompiledOperator (see attached picture) No matter use Nvida 3080TI or AMD 7900XTX, the problem is same.

image

Jay19751103 commented 1 month ago

Hi directML team

We need the fusion can work on SD3 text encoder 3. clone the issue from https://github.com/microsoft/onnxruntime/issues/21255