[Discussion] ORT GPU binaries do not contain DML

[gedoensmax]

Describe the issue

From what I understand DirectML is considered the default GPU backend on Windows systems. Nonetheless the "GPU" build does not contain DML. In my opinion it would make more sense to have the GPU package include DirectML besides CUDA and TRT. This package will then also run on any GPU. In addition such package can dynamically load OpenVino for additional GPU support on Intel.

To reproduce

When downloading a release package just call GetAvailableProviders(). This will report the supported providers. Another problem is that when dynamically loading OpenVINO it will never report it as supported provider.

Urgency

No response

Platform

Windows

OS Version

11

ONNX Runtime Installation

Released Package

ONNX Runtime Version or Commit ID

1.17.3

ONNX Runtime API

C++

Architecture

X64

Execution Provider

CUDA, DirectML, TensorRT

Execution Provider Library Version

No response

[gedoensmax]

@snnn or @pranavsharma can you help with this ?

[tenten8401]

+1 on this, currently making a custom build for DirectML support

Something I'm running into is that I can build it with CUDA or DirectML, but when I try both in the same build it fails because it can't find a bunch of dml stuff.. possibly related to why there are no combined builds by default?

.\build.bat --config RelWithDebInfo --parallel --skip_tests --build_shared_lib --compile_no_warning_as_error --use_cuda --cudnn_home "C:\Program Files\NVIDIA\CUDNN\v9.1" --cuda_home "C:\Program Files\NVIDIA GPU Computing Toolkit\CUDA\v12.4" --use_tensorrt --tensorrt_home "F:\Downloads\TensorRT-10.0.1.6" --use_dml

[saidwho12]

I agree with this, making a combined build would make things so much easier for me, in my software I want to dynamically switch between DirectML and TRT without needing to have 2 different EXEs