search

microsoft / onnxruntime

ONNX Runtime: cross-platform, high performance ML inferencing and training accelerator
https://onnxruntime.ai
MIT License
14.85k stars 2.94k forks source link

Support `__dlpack__` for OrtValues #22061

Open justinchuby opened 2 months ago

justinchuby commented 2 months ago

When users get the ort values and do stuff to them, what the experience will be: Currently they will call .numpy() and ortvalue_from_numpy() and some io binding if the tensor is on cuda. With __dlpack__ users can simply call torch.from_dlpack(ort_value) to get the cuda tensor in pytorch (and similar methods in other frameworks). No io binding setups needed.

Related: https://github.com/microsoft/onnxruntime/issues/15963

cc @yuslepukhin

github-actions[bot] commented 1 month ago

This issue has been automatically marked as stale due to inactivity and will be closed in 30 days if no further activity occurs. If further support is needed, please provide an update and/or more details.