OAID / Tengine

Tengine is a lite, high performance, modular inference engine for embedded device
Apache License 2.0
4.62k stars 999 forks source link

Cannot convert grouped convolution in ONNX? #1098

Open Nullkooland opened 3 years ago

Nullkooland commented 3 years ago

NN structure: ncc onnx

Log from convert_tool:

---- Tengine Convert Tool ---- 

Version     : v1.0, 14:33:02 Aug 31 2021
Status      : float32

----------onnx2tengine begin----------
Model op set is :13
Error: input elem num(148225) != reshaped elem num(4743200)
Tengine FATAL: Infer node(id: 12, op: Reshape) shape failed.
Convert model failed.

ONNX file: ncc.onnx.tar.gz

The Conv node is a grouped convolution with groups == 32, and the input tensor have 32 channels, so the output feature map's shape should be 1 x 32 x 385 x 385 == 4743200 elements. However, it seemed that the output tensor produced by tengine is actually in 1 x 1 x 385 x 385 == 148255 elements, so the subsequent Reshape Op failed because of elements number mismatch.

cccxinli commented 3 years ago

Hello, your model cannot be opened, could you check and re-upload it?

cccxinli commented 3 years ago

May fixed. Pull code and try again.

Nullkooland commented 3 years ago

The latest version convert tool is able to convert the model, but the feature map shape after convolution is still incorrect as I mentioned before. Seems that the groups attribute is ignored by the Conv node.

Converted tmfile structure: ncc tmfile

Model files: (original onnx and converted tmfile) ncc.tar.gz

Nullkooland commented 3 years ago

I guess the issue is with the tengine runtime, not the convert tool or the shape check (I guess you shouldn't have removed the shape check feature as it expose such problems early. Since the converted model is still incorrect and would trigger runtime error later in deployment environment.)

cccxinli commented 3 years ago

You are right. I just found the weight of this conv, is a variable tensor rather than a constant tensor.

Nullkooland commented 3 years ago

You are right. I just found the weight of this conv, is a variable tensor rather than a constant tensor.

Yes, what this NN do is to compute the normalized cross correlation between two feature maps, so the weights of the Conv is reshaped from an input tensor.