openvinotoolkit / openvino

OpenVINO™ is an open-source toolkit for optimizing and deploying AI inference
https://docs.openvino.ai
Apache License 2.0
6.85k stars 2.18k forks source link

[Bug]: Argument shapes are inconsistent #25649

Open ListonQH opened 2 months ago

ListonQH commented 2 months ago

OpenVINO Version

openvino - 2024.2.0

Operating System

Ubuntu 20.04 (LTS)

Device used for inference

CPU

Framework

PyTorch

Model used

No response

Issue description

Describe

When I try conver an ONNX to OV-IR, got error: Argument shapes are inconsistent.

ONNX as the picture: Screenshot from 2024-07-19 22-58-54

More

I also run it by onnxruntime and get target result:

import onnxruntime
import torch

ort_session = onnxruntime.InferenceSession("./test-error.onnx", providers=['CPUExecutionProvider'])
input = torch.rand(size=(1, 3, 20, 20))
onnxruntime_outputs = ort_session.run(None, {"input": input.numpy()})
print(onnxruntime_outputs)

Env

(openvino) lqh@lqh-pc:~/code/openvino_test$ pip list
Package            Version
------------------ ----------
coloredlogs        15.0.1
filelock           3.13.1
flatbuffers        24.3.25
fsspec             2024.2.0
humanfriendly      10.0
Jinja2             3.1.3
MarkupSafe         2.1.5
mpmath             1.3.0
networkx           3.2.1
numpy              1.26.3
onnx               1.16.1
onnxruntime        1.18.1
openvino           2024.2.0
openvino-telemetry 2024.1.0
packaging          24.1
pillow             10.2.0
pip                24.0
protobuf           5.27.2
setuptools         69.5.1
sympy              1.12
torch              2.3.1+cpu
torchaudio         2.3.1+cpu
torchvision        0.18.1+cpu
typing_extensions  4.9.0
wheel              0.43.0

Step-by-step reproduction

  1. python code get onnx file:
import torch

class TestModel(torch.nn.Module):
    def __init__(self, *args, **kwargs) -> None:
        super().__init__(*args, **kwargs)
        self.constan1 = torch.rand(size=(1, 3, 20, 20))

        self.tail_2 = torch.nn.Conv2d(in_channels=3, out_channels=1, kernel_size=3, stride=1)

    def forward(self, input):
        x = self.constan1 * input
        return self.tail_2(x)

m = TestModel()
m.eval()
torch.onnx.export(m, torch.rand(1, 3, 20, 20),f='test-error.onnx', input_names=['input'], output_names=['output'], do_constant_folding=False)
  1. ov.convert_model

when I try convert onnx to OV-IR:

import openvino as ov
m = ov.convert_model('test-error.onnx')
ov.save_model(m, 'test-error.xml')

Error:

Traceback (most recent call last):
  File "/home/lqh/code/openvino_test/ov_convert.py", line 3, in <module>
    m = ov.convert_model('test-error.onnx')
  File "/root/miniconda3/envs/openvino/lib/python3.10/site-packages/openvino/tools/ovc/convert.py", line 100, in convert_model
    ov_model, _ = _convert(cli_parser, params, True)
  File "/root/miniconda3/envs/openvino/lib/python3.10/site-packages/openvino/tools/ovc/convert_impl.py", line 534, in _convert
    raise e
  File "/root/miniconda3/envs/openvino/lib/python3.10/site-packages/openvino/tools/ovc/convert_impl.py", line 480, in _convert
    ov_model = driver(argv, {"conversion_parameters": non_default_params})
  File "/root/miniconda3/envs/openvino/lib/python3.10/site-packages/openvino/tools/ovc/convert_impl.py", line 241, in driver
    ov_model = moc_emit_ir(prepare_ir(argv), argv)
  File "/root/miniconda3/envs/openvino/lib/python3.10/site-packages/openvino/tools/ovc/moc_frontend/moc_emit_ir.py", line 20, in moc_emit_ir
    apply_moc_transformations(ngraph_function, cf=False, smart_reshape=True)
RuntimeError: Check 'TRShape::broadcast_merge_into(output_shape, input_shapes[1], autob)' failed at src/core/shape_inference/include/eltwise_shape_inference.hpp:26:
While validating node 'opset1::Multiply Multiply_1452 (opset1::Constant tail_2.weight[0]:f32[1,3,3,3], opset1::Constant /Constant[0]:f32[1,3,20,20]) -> (dynamic[...])' with friendly_name 'Multiply_1452':
Argument shapes are inconsistent.
  1. However, follow script running OK:
    
    import torch

class TestModel(torch.nn.Module): def init(self, *args, *kwargs) -> None: super().init(args, **kwargs) self.constan1 = torch.rand(size=(1, 4, 18, 18))

    self.tail_1 = torch.nn.Conv2d(in_channels=3, out_channels=4, kernel_size=3, stride=1)
    self.tail_2 = torch.nn.Conv2d(in_channels=4, out_channels=1, kernel_size=9, stride=1)

def forward(self, input):
    x = self.tail_1(input)
    x = self.constan1 * x
    return self.tail_2(x)

m = TestModel() m.eval() torch.onnx.export(m, torch.rand(1, 3, 20, 20),f='test.onnx', input_names=['input'], output_names=['output'], do_constant_folding=False)


ONNX: 

![Screenshot from 2024-07-19 23-10-30](https://github.com/user-attachments/assets/be15627c-c45a-4d5a-924f-d1a3b33e35d8)

### Relevant log output

_No response_

### Issue submission checklist

- [X] I'm reporting an issue. It's not a question.
- [X] I checked the problem with the documentation, FAQ, open issues, Stack Overflow, etc., and have not found a solution.
- [X] There is reproducer code and related data files such as images, videos, models, etc.
mvafin commented 1 month ago

Thank you for reporting this issue. It seems to be caused by fusing transformation. @itikhono please take a look.

CuriousPanCake commented 3 weeks ago

We're currently working on this issue. It requires some investigations and changes in Convolution transformations.

CuriousPanCake commented 3 weeks ago

The issue has been fixed https://github.com/openvinotoolkit/openvino/pull/25876 @ListonQH , please verify