OpenVINO 2024 not working properly

CSBVision commented 1 month ago

System Information

OpenCV Version: 4.10.0 Operating System / Platform: Windows 11 x64 Compiler & compiler version: VS 2022

Detailed description

With the 4.10.0 release, support for OpenVINO 2024 is included. Compiling and linking against OpenVINO 2024 works, however activating at runtime results in runtime error.s

Steps to reproduce

Compile OpenVINO 2024. We used 2024.2.0, only using the CPU plugin and compiled it as static libraries.
Compile OpenCV using OpenVINO, except those necessary for OpenVINO, further CMake configurations are not required.
Thereafter, run the following code:

import numpy as np
import cv2 as cv

# model is available in models/image_classification_mobilenet inside OpenCV's zoo repository
model = cv.dnn.readNet('image_classification_mobilenetv2_2022apr.onnx')

# use OpenVINO backend (no error if DNN_BACKEND_OPENCV is used instead)
model.setPreferableBackend(cv.dnn.DNN_BACKEND_INFERENCE_ENGINE)

# forward zero image
model.setInput(np.zeros((1,3,224,224), dtype=np.float32))
model.forward()

Here, forward() terminates with the following error:

error: OpenCV(4.10.0) opencv\modules\dnn\src\ie_ngraph.cpp:558: error: (-215:Assertion failed) blobIt != allBlobs.end() in function 'cv::dnn::InfEngineNgraphNet::forward'

As both, OpenCV's backend as well as OpenVINO 2023 work fine, the issue is caused by using OpenVINO 2024.

Issue submission checklist

[X] I report the issue, it's not a question
[X] I checked the problem with documentation, FAQ, open issues, forum.opencv.org, Stack Overflow, etc and have not found any solution
[X] I updated to the latest OpenCV version and the issue is still there
[X] There is reproducer code and related data files (videos, images, onnx, etc)

asmorkalov commented 1 month ago

@CSBVision Am I right that you do not use pre-built binaries of OpenVINO, but built it by yourself? Could you add build steps to the issue then?

CSBVision commented 1 month ago

Yes, correct, we compiled OpenVINO as static libraries for testing purposes, skipping most parts not required for OpenCV:

BUILD_SHARED_LIBS=False
ENABLE_TBBBIND_2_5=False
ENABLE_SYSTEM_TBB=True
TBB_DIR=/path/to/TBB
ENABLE_INTEL_CPU=True
ENABLE_OV_IR_FRONTEND=True
BUILD_template_extension=False
ENABLE_FASTER_BUILD=True
ENABLE_AUTO=False
ENABLE_AUTO_BATCH=False
ENABLE_HETERO=False
ENABLE_INTEL_GPU=False
ENABLE_INTEL_NPU=False
ENABLE_MULTI=False
ENABLE_SAMPLES=False
ENABLE_TEMPLATE=False
ENABLE_OV_ONNX_FRONTEND=False
ENABLE_OV_PADDLE_FRONTEND=False
ENABLE_OV_PYTORCH_FRONTEND=False
ENABLE_OV_TF_FRONTEND=False
ENABLE_OV_TF_LITE_FRONTEND=False
ENABLE_PYTHON=False
ENABLE_SAMPLES=False

We used this build configuration for both, 2023 and 2024 releases, and 2023 works fine whereas 2024 does not.

CSBVision commented 1 month ago

After your comment, we retested the official OpenVINO build. The following steps reproduce exactly the same issue, too, at least in all our tests:

Download OpenVINO 2024.2.0 and unpack the archive.
Clone OpenCV 4.10.0.
Run CMake with only these custom flags (assumes that Python is found automatically, otherwise extend it for your Python installation accordingly):
```
WITH_OPENVINO=True
OPENCV_DNN_OPENVINO=True
OpenVINO_DIR=/Path/to/OpenVINO/runtime/cmake
```
Compile OpenCV.
Setup the cv2 package (might require to copy the OpenVINO runtime DLLs to OpenCV's x64\vcXX\bin directory).
Use above Python code to reproduce the issue.

AvenSun commented 2 weeks ago

OpenCV (4.10.0) & OpenVINO (2024.3.0) has the same problem .

asmorkalov commented 1 week ago

Reproduced with current 4.x and fresh OpenVINO release w_openvino_toolkit_windows_2024.3.0.16041.1e3b88e4e3f_x86_64.

C++ code:

#include <iostream>
#include <opencv2/opencv.hpp>

void main()
{
    // model is available in models / image_classification_mobilenet inside OpenCV's zoo repository
    cv::dnn::Net model = cv::dnn::readNet("image_classification_mobilenetv2_2022apr.onnx");

    // use OpenVINO backend(no error if DNN_BACKEND_OPENCV is used instead)
    model.setPreferableBackend(cv::dnn::DNN_BACKEND_INFERENCE_ENGINE);

    // forward zero image

    cv::Mat input_image = cv::Mat::zeros(224, 224, CV_8UC3);
    cv::Mat input_blob = cv::dnn::blobFromImage(input_image);
    model.setInput(input_blob);
    printf("Before forward\n");
    model.forward();
    printf("After forward\n");
}

Error:

Before forward
OpenCV: terminate handler is called! The last OpenCV error is:
OpenCV(4.10.0-dev) Error: Assertion failed (blobIt != allBlobs.end()) in cv::dnn::InfEngineNgraphNet::forward, file D:\opencv\modules\dnn\src\ie_ngraph.cpp, line 558

asmorkalov commented 1 week ago

@dkurt could you take a look on it?

opencv / opencv