PINTO0309
commented
2 months ago I was aware of that possible problem when I implemented this feature. Whether you use concrete_function or not, the result is the same.
- conc
# INT8 Converter # converter = tf.lite.TFLiteConverter.from_saved_model( # output_folder_path, # ) converter = tf.lite.TFLiteConverter.from_concrete_functions( [concrete_func] ) - run
python onnx2tf.py -i sample.onnx -o outputs -coion -oiqt - result
Frankly, I can't think of a way to make the name order consistent with ONNX since the beginning of the creation of -coion. TensorFlow changes the order of inputs and outputs on its own. Therefore, there is virtually no information that ties the order of ONNX input/output names to the order of TFLite input/output names, and there is no way to reconcile that problem even if it were recognized that it occurs. Note that TensorFlow automatically generates garbage input/output names that I don't understand in the process of converting to TFLite, so any names/orders I specify in the process of generating Keras models are ignored.
From this TensorFlow behavior, I would guess that there is probably a bug in the FlatBuffer rewriting operation after INT8 quantization that is ignoring the input/output order. As evidence, the phenomenon is not reproduced for Float32, where TensorFlow does not need to rewrite FlatBuffer. This is talking about TensorFlow's internal processing, not -coion.
flat_subgraphs.inputs
array([0, 1], dtype=int32)
flat_subgraphs.outputs
array([2, 3], dtype=int32)
# It's mean
# location: 0 -> [1, 64, 64, 3] --- TensorFlow ignores input orders and misplaces shapes
# location: 1 -> [1, 128, 128, 3] --- TensorFlow ignores input orders and misplaces shapes
# location: 2 -> [1, 128, 128, 3]
# location: 3 -> [1, 64, 64, 3]
#
# Regardless of whether it is `concreate_func` or `saved_model`,
# the model input/output order is shuffled when processing begins in TensorFlow.
onnx_input_names
['input0', 'input1']
onnx_output_names
['output0', 'output1']To put the matter in proper perspective, the output order is not broken, but the input order is broken.
Since I have no control over the internal workings of TensorFlow myself, I am forced to include the following tutorial in the README. If you don't like the order of the names, just rewrite the names after generating the INT8 FlatBuffer.
ysohma
github-actions[bot]
Issue Type
Others
OS
Linux
onnx2tf version number
1.22.4
onnx version number
1.15.0
onnxruntime version number
1.17.1
onnxsim (onnx_simplifier) version number
0.4.33
tensorflow version number
2.16.1
Download URL for ONNX
N/A. provide by an attachment.
Parameter Replacement JSON
Description
When using the -coion option, the order of input and output names may sometimes swap.
Reproduction Steps
Prepare a simple model with two inputs (input0, input1) with different shapes for identification.
Convert the model using the following command:
Observe the converted models for both the floating-point model (float32) and the fully integer-quantized model (full_integer_quant)
outputs/sample_float32.tflite
outputs/sample_full_integer_quant.tflite
In the floating-point model, input0 and input1 are correctly reflected, but in the fully integer-quantized model, input0 and input1 are swapped.
This indicates that when using the -coion option with models that have multiple inputs and outputs, the fully integer-quantized model may have discrepancies in input and output names compared to the original ONNX model.
Analysis
To identify the cause of this issue, debug code was inserted into the onnx2tf source around here
It was found that the order of inputs/outputs in the flatbuffer does not match the order in the original ONNX model.
In the conversion of the floating-point model, the TFLiteConverter is created directly from the concrete_function. However, for the fully integer-quantized model, the TFLiteConverter is created from a SavedModel with signaturedefs. This difference in method causes the order mismatch.