Closed syjeon121 closed 3 months ago
I have not tried it before, but the GetD3D12ResourceFromAllocation method of the OrtDmlApi class should allow you to get the underlying ID3D12Resource from an OrtValue. You also have methods to provide inputs as GPU resources. The drawback is since it is an execution provider specific API, you would closely couple your code to DML.
Since some operators in DML are CPU based, I am also not sure that it will bring the speed up you expect. I think for speeding up DML first we would have to increase the parallelism of the graph executor in it. As for me it seems that due to aggressive memory fencing only one node might execute at a time, reducing compute performance, where some of the GPU might go under utilized.
Edit: note that currently I am using my own fork of ONNX / DML, where I have added significant performance improvements by decreasing and better managing the VRAM in the DirectML execution provider.
thank you for your advice. i will try :smile:
@axodox hi, thank you for your work I would like to use the GPU data of result image continuously without download to CPU. How to access GPU data of txt2img result?