search

intel / onnxruntime

ONNX Runtime: cross-platform, high performance scoring engine for ML models
MIT License
56 stars 22 forks source link

fix: updated data ops to support the complete graph on OVEP (#19975) #382

Closed ankitm3k closed 2 months ago

ankitm3k commented 2 months ago

Description

The ONNX model provided by issue author was not fully supported for OVEP and was failing inference with ort_perf_test app. The current PR enables GRU and LogSoftmax Op which helps enable the whole model graph on OVEP during execution. The unit test for GRU op is disabled.

Also investigating the inference output for multiple iterations for a single common input, the model was giving consistent and correct output across all the inference iterations during testing. Thus solving any post first inference regression of output for the given model architecture.

This PR fixes - https://github.com/microsoft/onnxruntime/issues/19975

preetha-intel commented 2 months ago

@ankitm3k The changes looks good. Have we verified the unit test cases and ensured there are no unit test failures for the enabled operators in CPU and NPU ?

ankitm3k commented 2 months ago

@vthaniel We have tested these changes on the CPU, GPU and it was working successfully earlier. I'd recommend before merging run the pipeline once to double check.

@preetha-intel We are not including these Ops for NPU now as the issue authors are using older machines and this is subject to support availability by OV toolkit also. For now, lets keep these ops supported for CPU and GPU only .

sfatimar commented 2 months ago

Did you check if any cpu C++ unit test/python test are failing with this ?

sfatimar commented 2 months ago

Please also run internal CI

sfatimar commented 2 months ago

LGTM.

vthaniel commented 2 months ago

@ankitm3k The CPU unit tests look good on ovep-develop-lnl-1.1 branch

ankitm3k commented 2 months ago

Thanks @vthaniel for verification. Lets upstream this too @preetha-intel today.