`get_device()` in onnxruntime-gpu not working properly

[Dr-Electron]

Describe the issue

Not sure if I understood get_device wrong, but It should give me the type of device that executed my model, right?

But I tried to use it with different execution providers (CPU/GPU) in onnxruntime-gpu and it always returns GPU

To reproduce

1) Install onnxruntime: pip install onnxruntime-gpu==1.17.1 2) Execute the following script

import onnxruntime

providers = ['CUDAExecutionProvider']

session = onnxruntime.InferenceSession('./model.onnx', providers=providers)
print(onnxruntime.get_device())

3) It should print GPU 4) Change the EP to CPU:

import onnxruntime

providers = ['CPUExecutionProvider']

session = onnxruntime.InferenceSession('./model.onnx', providers=providers)
print(onnxruntime.get_device())

Urgency

For now I find a workaround by using session.get_tuning_results() which I randomly found. Not sure if it is the best thing to use as it is not documented but until now it did what I needed 😅 I guess https://github.com/microsoft/onnxruntime/issues/16481 would be really nice to have

Platform

Windows

OS Version

21H2 (OS build 19044.4170)

ONNX Runtime Installation

Released Package

ONNX Runtime Version or Commit ID

1.17.1

ONNX Runtime API

Python

Architecture

X64

Execution Provider

Default CPU, CUDA

Execution Provider Library Version

CUDA 11.8

[hariharans29]

get_device() gives you the flavor of hardware acceleration offered by the installed package not which EP is used to run the session.

You can check - https://github.com/microsoft/onnxruntime/blob/4ea54b82f9debd70e46ea0a789e7aafe05d5b983/onnxruntime/python/onnxruntime_pybind_state.cc#L1210 and https://github.com/microsoft/onnxruntime/blob/4ea54b82f9debd70e46ea0a789e7aafe05d5b983/onnxruntime/python/onnxruntime_pybind_state_common.h#L26

[Dr-Electron]

Ok thanks. Is there a way to find out which EP my session is using?

[github-actions[bot]]

This issue has been automatically marked as stale due to inactivity and will be closed in 30 days if no further activity occurs. If further support is needed, please provide an update and/or more details.