QLinearConv breaks when bias is a 1-D tensor

[derekelkins]

Describe the bug

When QLinearConv is used with a 1-D bias, it is expecting the wrong shape. Specifically, given a 1x3x224x224 x input (NxCxHxW), a 64x3x7x7 w input (MxCxkHxkW) and a 64 B input (M) (with strides=[2,2] producing a 1x64x112x112 output) it fails with one of the following two assertions depending on whether w_scale is a scalar or a 1-D tensor (of length 64):

w_scale a scalar:

ValueError: Dimensions must be equal, but are 64 and 3 for '{{node truediv_1}} = RealDiv[T=DT_FLOAT](Cast_388, mul_193)' with input shapes: [64], [3].

w_scale a 1-D tensor:

ValueError: Dimensions must be equal, but are 112 and 64 for '{{node add_2}} = AddV2[T=DT_FLOAT](add_1, Round_1)' with input shapes: [1,64,112,112], [64].

To Reproduce

Run either of the attached models with a 1x3x224x224 input, e.g. with

import numpy as np
import onnx
from onnx_tf.backend import prepare

source_onnx = onnx.load("qlinear_conv_tensor_test.onnx")
# source_onnx = onnx.load("qlinear_conv_scalar_test.onnx")
source_tf_rep = prepare(source_onnx)
input = np.random.randn(1, 3, 224, 224)
source_tf_rep.run(input)

ONNX model file

qlinear_conv_test_cases.zip

Python, ONNX, ONNX-TF, Tensorflow version

• Python version: 3.9.7
• ONNX version: 1.11.0
• ONNX-TF version: 1.10.0
• Tensorflow version: tensorflow-cpu 2.8.0 (on MacOSX)

Additional context

Per the ONNX spec, QLinearConv takes in an x input with dimensions NxCxHxW and a w input with dimensions MxCxkHxkW (group=1 in this case). w_scale and w_zero_point can either be scalars or 1-D tensors of length M. B is an optional input but must be a 1-D tensor of length M if provided.

Looking at the code the w_scale parameter in the scalar case gets converted to a 1-D using the x.shape[1] which presumably would be C which is 3. I would expect something like w.shape[0]. The same should be used for w_zero_point a few lines above. If w_scale is a 1-D tensor with length 64 (M), then the failure occurs when the bias is added to the output. At this point y is 1x64x112x112 and B has length 64. The line of code presumably relies on broadcasting rules that would work when B is a scalar, but I can't imagine would ever do the right thing when B is not a scalar.

[vanIvan]

I do have the same problem:

ValueError: Dimensions must be equal, but are 4096 and 128 for '{{node Add_101}} = Add[T=DT_FLOAT](concat_22/concat, Add_101/y)' with input shapes: [?,4,?,4096], [128].

when has reshaping operation in some layers. At first I had successfully replaced reshape operation to split + concat, and it worked, but after some alterations to model, problem came back.

[PINTO0309]

I tested the conversion with my own tool called onnx2tf, which I created as an alternative to onnx-tensorflow, and it successfully converted QLinearConv with 1D bias. I am only interested in model conversions, not checking for accuracy degradation. https://github.com/PINTO0309/onnx2tf

onnx2tf -i qlinear_conv_tensor_test.onnx

Converted tflite https://github.com/PINTO0309/onnx2tf/releases/tag/1.1.20