[Bug]: openvino.tools.ovc.error.Error: Unknown model type: <class 'onnx.onnx_ml_pb2.ModelProto'>

[huangqiu15444]

OpenVINO Version

openvino 2024.3.0 + onnx 1.14.0

Operating System

Ubuntu 20.04 (LTS)

Device used for inference

CPU

Framework

ONNX

Model used

resnet18

Issue description

Model has been converted to ONNX format and saved as mobilenet_v2.onnx WARNING:nncf:ONNX models with 10 < opset version < 13 do not support per-channel quantization. Per-tensor quantization will be applied. Statistics collection ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 100% 1/1 • 0:00:00 • 0:00:00 Applying Fast Bias correction ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 100% 21/21 • 0:00:01 • 0:00:00 Traceback (most recent call last): File "nncf_tool.py", line 145, in ov_quantized_model = ov.convert_model(quantized_model) File "/home/huangq/anaconda3/envs/py38/lib/python3.8/site-packages/openvino/tools/ovc/convert.py", line 100, in convert_model ovmodel, = _convert(cli_parser, params, True) File "/home/huangq/anaconda3/envs/py38/lib/python3.8/site-packages/openvino/tools/ovc/convert_impl.py", line 548, in _convert raise e File "/home/huangq/anaconda3/envs/py38/lib/python3.8/site-packages/openvino/tools/ovc/convert_impl.py", line 448, in _convert model_framework = check_model_object(args) File "/home/huangq/anaconda3/envs/py38/lib/python3.8/site-packages/openvino/tools/ovc/convert_impl.py", line 241, in check_model_object raise Error('Unknown model type: {}'.format(type(model))) openvino.tools.ovc.error.Error: Unknown model type: <class 'onnx.onnx_ml_pb2.ModelProto'>

Step-by-step reproduction

if name == 'main':

onnx_quantization()

import torch
import torch.nn as nn
import torchvision.models as models
import onnx

# 1. 创建 MobileNetV2 模型
model = models.resnet18(pretrained=True)  # 使用预训练的 MobileNetV2 模型

# 2. 设置模型为评估模式
model.eval()

# 3. 创建一个虚拟输入张量
# 假设输入图像的大小为 224x224，批量大小为 1
dummy_input = torch.randn(1, 3, 224, 224)

# 4. 定义 ONNX 导出路径
onnx_file_path = "mobilenet_v2.onnx"

# 5. 导出模型到 ONNX 格式
torch.onnx.export(
    model,                # 要导出的模型
    dummy_input,          # 输入张量
    onnx_file_path,       # 导出的 ONNX 文件路径
    verbose=True,         # 是否打印导出过程的信息
    input_names=['input'],# 输入张量的名称
    output_names=['output'], # 输出张量的名称
    opset_version=12      # ONNX 操作集版本
)

print(f"Model has been converted to ONNX format and saved as {onnx_file_path}")

from torchvision import datasets, transforms

# Instantiate your uncompressed model
onnx_model = onnx.load_model(onnx_file_path)

# Provide validation part of the dataset to collect statistics needed for the compression algorithm
val_dataset = datasets.ImageFolder("/home/huangq/nfs/common_train_tools/qunat/", transform=transforms.Compose([transforms.Resize((224, 224)), transforms.ToTensor()]))
dataset_loader = torch.utils.data.DataLoader(val_dataset, batch_size=1)

# Step 1: Initialize transformation function
input_name = onnx_model.graph.input[0].name
def transform_fn(data_item):
    images, _ = data_item
    return {input_name: images.numpy()}

# Step 2: Initialize NNCF Dataset
calibration_dataset = nncf.Dataset(dataset_loader, transform_fn)
# Step 3: Run the quantization pipeline
quantized_model = nncf.quantize(onnx_model, calibration_dataset)

ov_quantized_model = ov.convert_model(quantized_model)

Relevant log output

No response

Issue submission checklist

• [X] I'm reporting an issue. It's not a question.
• [X] I checked the problem with the documentation, FAQ, open issues, Stack Overflow, etc., and have not found a solution.
• [X] There is reproducer code and related data files such as images, videos, models, etc.

[andrei-kochin]

Hello @huangqiu15444,

Thank you for reaching the OpenVINO! Conversion doesn't support the ONNX ModelProto object while only supports the BytesIO python object. Before passing this model you have to serialize it and only then pass .onnx file to the convert model.

Best regards, Andrei

[andrei-kochin]

@huangqiu15444 have you managed to try?

[ravizhan]

I got the same error. According to @andrei-kochin , I try to use the following code, and it does work, but I am not sure if it was correct or not.

from io import BytesIO
model = onnx.load("siamese.onnx")
calibration_dataset = nncf.Dataset(calibration_loader, transform_fn)
quantized_model = nncf.quantize(model, calibration_dataset, preset=nncf.QuantizationPreset.MIXED, fast_bias_correction=False)

ov_quantized_model = ov.convert_model(BytesIO(quantized_model.SerializeToString()))

model_int8 = ov.compile_model(ov_quantized_model)

ov.save_model(ov_quantized_model, "quantized_model.xml")

[ravizhan]

BTW, I notice that the code one the doc isn't seem correct. When running it, it occur the same error above.

[andrei-kochin]

@ravizhan if you run the code snippet you've listed you shouldn't get the 'onnx.onnx_ml_pb2.ModelProto' object so the error should be different.

I've tried your code and saw only a warning but but only for the first time. Second time it was fine without any warning:

>>> ov_quantized_model = ov.convert_model(BytesIO(quantized_model.SerializeToString()))
WARNING:tensorflow:From C:\Users\akochin\AppData\Roaming\Python\Python312\site-packages\openvino\frontend\tensorflow\utils.py:388: The name tf.GraphDef is deprecated. Please use tf.compat.v1.GraphDef instead.

[ WARNING ]  From %s: The name %s is deprecated. Please use %s instead.

>>> ov_quantized_model = ov.convert_model(BytesIO(quantized_model.SerializeToString()))

Could you please tell me what error do you see?

Typical scenario for the BytesIO is when onnx model was exported from torch model so we definitely support this path.

[ravizhan]

Of course. I have tried siamese.onnx and yolov8.onnx with the same code. Both of the siamese.onnx and yolov8.onnx are export from pytorch. And they occur the same error.

[avitial]

@huangqiu15444 please try the onnx port changes as listed in the PR above, using your code snippet the issue disappeared by adding the following lines. Note I tried this with openvino-nightly== 2024.5.0.dev20240911and cannot see the error. Closing this. Please try on your side, feel free to reopen and ask additional questions if issue persists.

[...]
#ov_quantized_model = ov.convert_model(quantized_model)
# use a temporary file to convert ONNX model to OpenVINO model
quantized_model_path = "quantized_model.onnx"
onnx.save(quantized_model, quantized_model_path)

ov_quantized_model = ov.convert_model(quantized_model_path)