search

open-mmlab / mmdeploy

OpenMMLab Model Deployment Framework
https://mmdeploy.readthedocs.io/en/latest/
Apache License 2.0
2.76k stars 632 forks source link

[Feature] Code update for supporting latest onnxruntime & OpenVINO #2779

Open RiverLight4 opened 4 months ago

RiverLight4 commented 4 months ago

Motivation

MMDeploy will not be able to convert to OpenVINO model at the end of 2024, because onnxruntime and OpenVINO runtime supported by MMDeploy 2.3.0 will be End-of-life.

MMdeploy must be updated to support latest OpenVINO and onnxruntime which is officially supported ASAP.

Related resources

MMDeploy 2.3.0 requires onnxruntime <= v1.15.1 because MMDeploy SDK currently rely on Ort::CustomOpApi removed at v1.16.0. (see https://github.com/open-mmlab/mmdeploy/issues/2465 ) However, it seems that there is no plan to follow onnxruntime's update now.

onnxruntime v1.15.1 only supports OpenVINO 2022.x and 2023.0. (Please see CMakeLists.txt https://github.com/microsoft/onnxruntime/blob/baeece44ba075009c6bfe95891a8c1b3d4571cb3/cmake/CMakeLists.txt#L1154 ) Even if changing CMakeLists.txt and run, it isn't work properly. onnxruntime v1.15.1 cannot be used with OpenVINO 2024.x because some APIs are deprecated and removed. (InferenceEngine and ngraph)

OpenVINO regular release and LTS release policy is found here: https://docs.openvino.ai/nightly/about-openvino/release-notes-openvino/release-policy.html

(bolded by my hand)

Each regular release is supported until the next version arrives, making it suitable for: Most typical use cases (the recommended release type). Products requiring frequent changes in supported hardware, libraries, operating systems, and models.

LTS Lifecycle LTS is typically published at the end of every year cycle. LTS uses the branch of the last yearly regular release. LTS aim to receive an update once a year. Security updates are offered for the duration of the entire LTS period, which is two years (or until superseded by two consecutive LTS versions). Updates targeting newly discovered bugs are offered for the period of one year.

It says that OpenVINO 2022.x is already EOL. and even OpenVINO 2022.3 LTS will be outdated at the end of 2024.

Additional context

No response

RiverLight4 commented 3 months ago

This problem might be resolved temporarily when PR https://github.com/open-mmlab/mmdeploy/pull/2755 is merged successfully. Could you please review and merge it?