search

PINTO0309 / onnx2tf

Self-Created Tools to convert ONNX files (NCHW) to TensorFlow/TFLite/Keras format (NHWC). The purpose of this tool is to solve the massive Transpose extrapolation problem in onnx-tensorflow (onnx-tf). I don't need a Star, but give me a pull request.
MIT License
712 stars 73 forks source link

ValueError: Exception encountered when calling layer "tf.math.multiply_46" (type TFOpLambda). #569

Closed MuffinTopSJY closed 7 months ago

MuffinTopSJY commented 10 months ago

Issue Type

Others

OS

Linux

onnx2tf version number

1.19.5

onnx version number

1.15.0

onnxruntime version number

1.16.3

onnxsim (onnx_simplifier) version number

0.4.33

tensorflow version number

2.15.0

Download URL for ONNX

-

Parameter Replacement JSON

-

Description

Hi, sorry to bother you again.

Following https://github.com/PINTO0309/LightGlue-ONNX, I tried converting Lightglue to Onnx model by python export.py --img_size 512 512 --lightglue_path weights/sjy_fused_static.onnx --end2end, and the Onnx model worked in my test.

When converting by onnx2tf -i sjy_fused_static.onnx or onnx2tf -i sjy_fused_static.onnx -ois image0:1,1,512,512 image1:1,1,512,512, (in superpoint.py, I set top_num = 300) I got this error: 

INFO: 1409 / 3391
INFO: onnx_op_type: Expand onnx_op_name: /lightglue/posenc/Expand
INFO:  input_name.1: /lightglue/posenc/Unsqueeze_3_output_0 shape: [2, 1, 1, 300, 32, 1] dtype: float32
INFO:  input_name.2: /lightglue/posenc/Where_output_0 shape: [6] dtype: int64
INFO:  output_name.1: /lightglue/posenc/Expand_output_0 shape: [2, 1, 1, 300, 32, 2] dtype: float32
ERROR: The trace log is below.
Traceback (most recent call last):
  File "/home/feiluo/.conda/envs/onnx2tf/lib/python3.10/site-packages/onnx2tf/utils/common_functions.py", line 310, in print_wrapper_func
    result = func(*args, **kwargs)
  File "/home/feiluo/.conda/envs/onnx2tf/lib/python3.10/site-packages/onnx2tf/utils/common_functions.py", line 383, in inverted_operation_enable_disable_wrapper_func
    result = func(*args, **kwargs)
  File "/home/feiluo/.conda/envs/onnx2tf/lib/python3.10/site-packages/onnx2tf/utils/common_functions.py", line 53, in get_replacement_parameter_wrapper_func
    func(*args, **kwargs)
  File "/home/feiluo/.conda/envs/onnx2tf/lib/python3.10/site-packages/onnx2tf/ops/Expand.py", line 118, in make_node
    expanded_tensor = input_tensor * ones
  File "/home/feiluo/.conda/envs/onnx2tf/lib/python3.10/site-packages/tensorflow/python/util/traceback_utils.py", line 153, in error_handler
    raise e.with_traceback(filtered_tb) from None
  File "/home/feiluo/.conda/envs/onnx2tf/lib/python3.10/site-packages/keras/src/layers/core/tf_op_layer.py", line 119, in handle
    return TFOpLambda(op)(*args, **kwargs)
  File "/home/feiluo/.conda/envs/onnx2tf/lib/python3.10/site-packages/keras/src/utils/traceback_utils.py", line 70, in error_handler
    raise e.with_traceback(filtered_tb) from None
ValueError: Exception encountered when calling layer "tf.math.multiply_46" (type TFOpLambda).

Dimensions must be equal, but are 32 and 2 for '{{node tf.math.multiply_46/Mul}} = Mul[T=DT_FLOAT](Placeholder, tf.math.multiply_46/Mul/y)' with input shapes: [1,2,1,300,32,1], [1,1,1,1,2,1].

Call arguments received by layer "tf.math.multiply_46" (type TFOpLambda):
  • x=tf.Tensor(shape=(1, 2, 1, 300, 32, 1), dtype=float32)
  • y=tf.Tensor(shape=(1, 1, 1, 1, 2, 1), dtype=float32)
  • name=None

The structure diagram of sjy_fused_static.onnx related to onnx_op_name: /lightglue/posenc/Expand might be like this:

image

Besides, I have doubts about these results, the output.shape of tf doesn't match the output of onnx. In tf_op_type: Concat, it seems that input1 and input2 concat on axis=1, I don't know what it is supposed to be or if it matters, for example:

INFO: 1401 / 3391
INFO: onnx_op_type: Concat onnx_op_name: /lightglue/posenc/Concat
INFO:  input_name.1: /lightglue/posenc/Unsqueeze_output_0 shape: [1, 1, 300, 32] dtype: float32
INFO:  input_name.2: /lightglue/posenc/Unsqueeze_1_output_0 shape: [1, 1, 300, 32] dtype: float32
INFO:  output_name.1: /lightglue/posenc/Concat_output_0 shape: [2, 1, 300, 32] dtype: float32
INFO: tf_op_type: concat
INFO:  input.1.input0: name: tf.reshape_36/Reshape:0 shape: (1, 1, 300, 32) dtype: <dtype: 'float32'> 
INFO:  input.2.input1: name: tf.reshape_37/Reshape:0 shape: (1, 1, 300, 32) dtype: <dtype: 'float32'> 
INFO:  input.3.axis: val: 0 
INFO:  output.1.output: name: tf.concat_29/concat:0 shape: (1, 2, 300, 32) dtype: <dtype: 'float32'> 

INFO: 1402 / 3391
INFO: onnx_op_type: Concat onnx_op_name: /lightglue/posenc_1/Concat
INFO:  input_name.1: /lightglue/posenc_1/Unsqueeze_output_0 shape: [1, 1, 300, 32] dtype: float32
INFO:  input_name.2: /lightglue/posenc_1/Unsqueeze_1_output_0 shape: [1, 1, 300, 32] dtype: float32
INFO:  output_name.1: /lightglue/posenc_1/Concat_output_0 shape: [2, 1, 300, 32] dtype: float32
INFO: tf_op_type: concat
INFO:  input.1.input0: name: tf.reshape_38/Reshape:0 shape: (1, 1, 300, 32) dtype: <dtype: 'float32'> 
INFO:  input.2.input1: name: tf.reshape_39/Reshape:0 shape: (1, 1, 300, 32) dtype: <dtype: 'float32'> 
INFO:  input.3.axis: val: 0 
INFO:  output.1.output: name: tf.concat_35/concat:0 shape: (1, 2, 300, 32) dtype: <dtype: 'float32'> 

INFO: 1405 / 3391
INFO: onnx_op_type: Unsqueeze onnx_op_name: /lightglue/posenc/Unsqueeze_3
INFO:  input_name.1: /lightglue/posenc/Concat_output_0 shape: [2, 1, 300, 32] dtype: float32
INFO:  input_name.2: 4856 shape: [2] dtype: int64
INFO:  output_name.1: /lightglue/posenc/Unsqueeze_3_output_0 shape: [2, 1, 1, 300, 32, 1] dtype: float32
INFO: tf_op_type: reshape
INFO:  input.1.tensor: name: tf.concat_29/concat:0 shape: (1, 2, 300, 32) dtype: <dtype: 'float32'> 
INFO:  input.2.shape: val: [1, 2, 1, 300, 32, 1] 
INFO:  output.1.output: name: tf.reshape_40/Reshape:0 shape: (1, 2, 1, 300, 32, 1) dtype: <dtype: 'float32'> 

INFO: 1406 / 3391
INFO: onnx_op_type: Unsqueeze onnx_op_name: /lightglue/posenc_1/Unsqueeze_3
INFO:  input_name.1: /lightglue/posenc_1/Concat_output_0 shape: [2, 1, 300, 32] dtype: float32
INFO:  input_name.2: 4856 shape: (2,) dtype: int64
INFO:  output_name.1: /lightglue/posenc_1/Unsqueeze_3_output_0 shape: [2, 1, 1, 300, 32, 1] dtype: float32
INFO: tf_op_type: reshape
INFO:  input.1.tensor: name: tf.concat_35/concat:0 shape: (1, 2, 300, 32) dtype: <dtype: 'float32'> 
INFO:  input.2.shape: val: [1, 2, 1, 300, 32, 1] 
INFO:  output.1.output: name: tf.reshape_41/Reshape:0 shape: (1, 2, 1, 300, 32, 1) dtype: <dtype: 'float32'>

I tried to figure it out but I'm sooo stupid. T_T Sincerely thank you for your time again.

PINTO0309 commented 7 months ago

https://github.com/PINTO0309/spo4onnx

python export.py \
--img_size 512 512 \
--lightglue_path weights/sjy_fused_static.onnx \
--end2end

pip install -U spo4onnx==1.0.4 onnx2tf==1.20.0 onnxsim==0.4.30

cd weights
spo4onnx -if sjy_fused_static.onnx -of sjy_fused_static_spo.onnx

onnx2tf -i sjy_fused_static_spo.onnx

image