Add model_priority as a provider option

intel / onnxruntime

ONNX Runtime: cross-platform, high performance scoring engine for ML models

MIT License

56 stars 22 forks source link

Closed sspintel closed 4 months ago

sspintel commented 4 months ago

High-level OpenVINO model priority hint. Defines what model should be provided with more performant bounded resource first.

It's an optional parameter to provide a hint to the scheduler if a workload has higher or lower QoS needs.

Valid values are: LOW, MEDIUM, HIGH, DEFAULT

sspintel commented 4 months ago

Merging PR to kick-off automated builds