Is support for ROCMExecutionProvider planned?

roboflow / inference

A fast, easy-to-use, production-ready inference server for computer vision supporting deployment of many popular model architectures and fine-tuned models.

https://inference.roboflow.com

Other

1.36k stars 125 forks source link

Is support for ROCMExecutionProvider planned? #510

Closed ak1932 closed 1 week ago

ak1932 commented 4 months ago

Search before asking

[X] I have searched the Inference issues and found no similar feature requests.

Description

Support for ROCMExecutionProvider would be great

Use case

This would make roboflow more accessible with users using amd gpu benefiting greatly.

Additional

No response

Are you willing to submit a PR?

[ ] Yes I'd like to help by submitting a PR!

ak1932 commented 4 months ago

Onnxruntime has stable onnxruntime-rocm packages (though not on PyPi). Also is there any way I can temporily use ROCMExecutionProvider?

yeldarby commented 4 months ago

We don't plan to officially support this at the moment because we don't have any customers that need/use it and don't have any hardware to test with.

But you should be able to get hardware acceleration on ROCM using the onnxruntime-rocm package by installing it manually. It should work just like the onnxruntime-silicon runtime does for MacOS acceleration.

Once installed, use the ONNXRUNTIME_EXECUTION_PROVIDERS="[ROCMExecutionProvider, CPUExecutionProvider]" environment variable to ensure that it gets picked up & used.

ak1932 commented 4 months ago

Ohh ok. I'll try it out. Thanks for the help.