yolov8n onnx to tflite without transpose ops

[codewarrior26]

Issue Type

Others

OS

Linux

onnx2tf version number

1.25.7

onnx version number

1.16.1

onnxruntime version number

1.18.1

onnxsim (onnx_simplifier) version number

0.4.33

tensorflow version number

2.17.0

Download URL for ONNX

https://drive.google.com/file/d/1aFK3NAkWutnOFFw2LWiun9JeYS8k0Qu6/view?usp=sharing

Parameter Replacement JSON

{
  "format_version": 1,
  "operations": [

    {
      "op_name": "wa/model.22/Concat",
      "param_target": "outputs",
      "param_name": "wa/model.22/Concat_output_0",
      "post_process_transpose_perm": [0, 3, 1, 2]
    }

  ]
}

Description

Purpose: Personal development

What: There are no errors occurring during the conversion from onnx to tflite, but I want to understand if there is a way to remove the one or more of (4) transpose operators in the resulting tflite graph during the conversion process.

How I used ultralytics library to convert yolov8n.pt model to tflite file using their export function. However, the final tflite file contains 4 transpose operators. It seems like ultralytics export function uses the onnx2tf tool for conversion, but it looks like they don't use --prf option. So, I want to use the onnx2tf to convert the yolov8n.onnx file to tflite float 32 by myself. So when I run !onnx2tf -i yolov8n.onnx, the conversion is successful, but I wanted to explore further if the 4 transpose ops can be avoided. So I wanted to understand how I could go about using the replacement.json file for achieving this.

For example: this is a part of the onnx graph (visualized on netron)

And for this part, the resulting tflite graph looks like:

where a transpose operator gets added between concatenation and reshape.

In order to avoid the above transpose operator, I attempted to use replace.json as stated above. But I'm pretty sure that won't help. I would really appreciate some help and guidance as to how I could approach avoiding the 4 transpose operators.

Why: If I can understand how to use replacement.json in this case, I could use that knowledge for conversion of similar models.

[PINTO0309]

Quite simply, the design of the model is not good enough. Unnecessary Reshape should not be used. The conversion to NHWC is not limited to YOLOv8, but other models always require a conversion to NCHW just before Reshape. This should be obvious if you understand the Reshape specification and its behavior. Before considering the use of replace.json, the first thing to do is to review the model design and remove unnecessary Reshape OPs from the model design.

If the model's transformation behavior is changed in replace.json so that it does not produce the Transpose immediately before the Reshape, the final output of the model will be severely corrupted because the model structure will be broken.

The JSON you created is not wrong. However, if you decide that Transpose is necessary, onnx2tf will automatically extrapolate Transpose because onnx2tf has an automatic correction feature that automatically corrects for erroneous tensors that degrade the accuracy of the model and automatically maintains the integrity of the final output of the model.

In other words, the process performed inside onnx2tf is as follows.

1. Disable Transpose via replace.json
2. Detect tensor corruption by Reshape and extrapolate Transpose automatically

In other words, onnx2tf tolerates few erroneous conversions that would cause accuracy degradation.

[codewarrior26]

Thank you very much for clarifying!