search

intel / onnxruntime

ONNX Runtime: cross-platform, high performance scoring engine for ML models
MIT License
56 stars 22 forks source link

Preetha/device precision change #358

Closed preetha-intel closed 4 months ago

preetha-intel commented 4 months ago

Description

Update the cli option for OVEP for setting device and inference precision as seperate components.

Motivation and Context

The model can be inferenced with following precisions:

To facilitate the user in selecting the preferred precision this PR introduces a new session option "precision". The usage will be to specify "device_type" session option with supported HW options from CPU, GPU and NPU. Supported values for "precision" session option will be FP32, FP16, ACCURACY. If precision is not specified, default optimized precision will be chosen for the specified hardware device.

device_id session option is deprecated

preetha-intel commented 4 months ago

Merging as its internally reviewed. This feature is critical to kick start validation pipelines.