search

microsoft / onnxruntime

ONNX Runtime: cross-platform, high performance ML inferencing and training accelerator
https://onnxruntime.ai
MIT License
14.6k stars 2.92k forks source link

Memory management #12824

Open Joeyzhouqihui opened 2 years ago

Joeyzhouqihui commented 2 years ago

Hi sorry for bothering. I am just wondering when using multiple streams for model inference. Will the cached memory blocks been shared by different streams? Or they are not shared like in pytorch?

hariharans29 commented 2 years ago

Please fill in the Github issue template providing more details of your use-case.