search

microsoft / onnxruntime

ONNX Runtime: cross-platform, high performance ML inferencing and training accelerator
https://onnxruntime.ai
MIT License
14.87k stars 2.94k forks source link

Stateful/Memory models #20943

Open bhack opened 6 months ago

bhack commented 6 months ago

Discussed in https://github.com/microsoft/onnxruntime/discussions/20619

Originally posted by **bhack** May 9, 2024 How we are going to support stateful model in onnxruntime? The majority of video segmentation models are going to manage an internal memory inside the same inference session: https://github.com/hkchengrex/XMem https://github.com/hkchengrex/Cutie https://github.com/yoxu515/aot-benchmark/ etc.. See also the thread at: https://github.com/pytorch/serve/issues/2743#issuecomment-1784262998
yuslepukhin commented 5 months ago

GenAI

github-actions[bot] commented 4 months ago

This issue has been automatically marked as stale due to inactivity and will be closed in 30 days if no further activity occurs. If further support is needed, please provide an update and/or more details.

bhack commented 4 months ago

Is there any interest in the topic?

bhack commented 4 months ago

See also how eventually they are going to workaround this in sam2: https://github.com/facebookresearch/segment-anything-2/issues/3