Closed glenn-jocher closed 1 year ago
convert - first try
openvino2tensorflow \
--model_path yolov5n.xml \
--output_saved_model \
--output_pb \
--non_verbose \
--output_no_quant_float32_tflite
Here. Shape Error
occurred.
ERROR: axes don't match array
ERROR: model_path : yolov5n.xml
ERROR: weights_path: yolov5n.bin
ERROR: layer_id : 271 <<<<<<<<<<<--------------------- Shape Error occured
ERROR: input_layer0 layer_id=269: KerasTensor(type_spec=TensorSpec(shape=(1, 80, 3, 85, 2), dtype=tf.float32, name=None), name='tf.math.multiply/Mul:0', description="created by layer 'tf.math.multiply'")
ERROR: input_layer1 layer_id=270: Const(ndarray).shape (1, 3, 80, 80, 2)
array([[[[[ -4., -4.],
[ 4., -4.],
[ 12., -4.],
:
:
ERROR: Please refer to 6-7 in the README first. https://github.com/PINTO0309/openvino2tensorflow
The conversion of the unpredictable transposition operation from NCDHW
to NCHWD
fails at this point. Mechanically unpredictable transposition operations require the tool's behavior to be modified using JSON files. The cause of this error is that the transpose before the OP in which the error occurs failed to correct the transposition order.
layer_type: Reshape
layer_id: 261
input_layer0: layer_id=259: KerasTensor(type_spec=TensorSpec(shape=(1, 80, 80, 255), dtype=tf.float32, name=None), name='tf.math.add_52/Add:0', description="created by layer 'tf.math.add_52'")
input_layer1_shape: layer_id=260: (5,)
tf_layers_dict: KerasTensor(type_spec=TensorSpec(shape=(1, 80, 80, 3, 85), dtype=tf.float32, name=None), name='tf.reshape/Reshape:0', description="created by layer 'tf.reshape'")
====================================================================================
layer_type: Const
layer_id: 262
tf_layers_dict_shape: (5,)
====================================================================================
layer_type: Transpose
layer_id: 263 <<<<<<<<<<<--------------------- The conversion of the unpredictable transposition operation from NCDHW to NCHWD fails at this point.
input_layer0: layer_id=261: KerasTensor(type_spec=TensorSpec(shape=(1, 80, 80, 3, 85), dtype=tf.float32, name=None), name='tf.reshape/Reshape:0', description="created by layer 'tf.reshape'")
input_layer1_shape: layer_id=262: (5,)
tf_layers_dict: KerasTensor(type_spec=TensorSpec(shape=(1, 80, 3, 85, 80), dtype=tf.float32, name=None), name='tf.compat.v1.transpose/transpose:0', description="created by layer 'tf.compat.v1.transpose'")
replace_n.json - first step
Rewrite the transposition order of Transpose
so that it behaves as expected. The inverted order constant corresponds to the layer number of the name of output minus 1. output name: 263:2
-> layer_id:262 = custom name: Constant_3067
{
"format_version": 2,
"layers": [
{
"layer_id": "262",
"type": "Const",
"replace_mode": "direct",
"values": [
0,
3,
1,
2,
4
]
}
]
}
re-convert
openvino2tensorflow \
--model_path yolov5n.xml \
--output_saved_model \
--output_pb \
--non_verbose \
--output_no_quant_float32_tflite \
--weight_replacement_config replace_n.json
Shape Error
occurred.
ERROR: axes don't match array
ERROR: model_path : yolov5n.xml
ERROR: weights_path: yolov5n.bin
ERROR: layer_id : 328 <<<<<<<<<<<--------------------- Shape Error occured
ERROR: input_layer0 layer_id=326: KerasTensor(type_spec=TensorSpec(shape=(1, 40, 3, 85, 2), dtype=tf.float32, name=None), name='tf.math.multiply_3/Mul:0', description="created by layer 'tf.math.multiply_3'")
ERROR: input_layer1 layer_id=327: Const(ndarray).shape (1, 3, 40, 40, 2)
array([[[[[ -8., -8.],
[ 8., -8.],
[ 24., -8.],
The conversion of the unpredictable transposition operation from NCDHW
to NCHWD
fails at this point.
====================================================================================
layer_type: Reshape
layer_id: 318
input_layer0: layer_id=316: KerasTensor(type_spec=TensorSpec(shape=(1, 40, 40, 255), dtype=tf.float32, name=None), name='tf.math.add_60/Add:0', description="created by layer 'tf.math.add_60'")
input_layer1_shape: layer_id=317: (5,)
tf_layers_dict: KerasTensor(type_spec=TensorSpec(shape=(1, 40, 40, 3, 85), dtype=tf.float32, name=None), name='tf.reshape_2/Reshape:0', description="created by layer 'tf.reshape_2'")
====================================================================================
layer_type: Const
layer_id: 319
tf_layers_dict_shape: (5,)
====================================================================================
layer_type: Transpose
layer_id: 320 <<<<<<<<<<<--------------------- The conversion of the unpredictable transposition operation from NCDHW to NCHWD fails at this point.
input_layer0: layer_id=318: KerasTensor(type_spec=TensorSpec(shape=(1, 40, 40, 3, 85), dtype=tf.float32, name=None), name='tf.reshape_2/Reshape:0', description="created by layer 'tf.reshape_2'")
input_layer1_shape: layer_id=319: (5,)
tf_layers_dict: KerasTensor(type_spec=TensorSpec(shape=(1, 40, 3, 85, 40), dtype=tf.float32, name=None), name='tf.compat.v1.transpose_1/transpose:0', description="created by layer 'tf.compat.v1.transpose_1'")
====================================================================================
replace_n.json - second step
{
"format_version": 2,
"layers": [
{
"layer_id": "262",
"type": "Const",
"replace_mode": "direct",
"values": [
0,
3,
1,
2,
4
]
},
{
"layer_id": "319",
"type": "Const",
"replace_mode": "direct",
"values": [
0,
3,
1,
2,
4
]
}
]
}
re-convert
openvino2tensorflow \
--model_path yolov5n.xml \
--output_saved_model \
--output_pb \
--non_verbose \
--output_no_quant_float32_tflite \
--weight_replacement_config replace_n.json
Shape Error
occurred.
ERROR: axes don't match array
ERROR: model_path : yolov5n.xml
ERROR: weights_path: yolov5n.bin
ERROR: layer_id : 384
ERROR: input_layer0 layer_id=382: KerasTensor(type_spec=TensorSpec(shape=(1, 20, 3, 85, 2), dtype=tf.float32, name=None), name='tf.math.multiply_6/Mul:0', description="created by layer 'tf.math.multiply_6'")
ERROR: input_layer1 layer_id=383: Const(ndarray).shape (1, 3, 20, 20, 2)
array([[[[[-16., -16.],
[ 16., -16.],
The conversion of the unpredictable transposition operation from NCDHW
to NCHWD
fails at this point.
====================================================================================
layer_type: Reshape
layer_id: 374
input_layer0: layer_id=372: KerasTensor(type_spec=TensorSpec(shape=(1, 20, 20, 255), dtype=tf.float32, name=None), name='tf.math.add_68/Add:0', description="created by layer 'tf.math.add_68'")
input_layer1_shape: layer_id=373: (5,)
tf_layers_dict: KerasTensor(type_spec=TensorSpec(shape=(1, 20, 20, 3, 85), dtype=tf.float32, name=None), name='tf.reshape_4/Reshape:0', description="created by layer 'tf.reshape_4'")
====================================================================================
layer_type: Const
layer_id: 375
tf_layers_dict_shape: (5,)
====================================================================================
layer_type: Transpose
layer_id: 376 <<<<<<<<<<<--------------------- The conversion of the unpredictable transposition operation from NCDHW to NCHWD fails at this point.
input_layer0: layer_id=374: KerasTensor(type_spec=TensorSpec(shape=(1, 20, 20, 3, 85), dtype=tf.float32, name=None), name='tf.reshape_4/Reshape:0', description="created by layer 'tf.reshape_4'")
input_layer1_shape: layer_id=375: (5,)
tf_layers_dict: KerasTensor(type_spec=TensorSpec(shape=(1, 20, 3, 85, 20), dtype=tf.float32, name=None), name='tf.compat.v1.transpose_2/transpose:0', description="created by layer 'tf.compat.v1.transpose_2'")
====================================================================================
replace_n.json - second step
{
"format_version": 2,
"layers": [
{
"layer_id": "262",
"type": "Const",
"replace_mode": "direct",
"values": [
0,
3,
1,
2,
4
]
},
{
"layer_id": "319",
"type": "Const",
"replace_mode": "direct",
"values": [
0,
3,
1,
2,
4
]
},
{
"layer_id": "375",
"type": "Const",
"replace_mode": "direct",
"values": [
0,
3,
1,
2,
4
]
}
]
}
re-convert
openvino2tensorflow \
--model_path yolov5n.xml \
--output_saved_model \
--output_pb \
--non_verbose \
--output_no_quant_float32_tflite \
--weight_replacement_config replace_n.json
success
====================================================================================
layer_type: Result
layer_id: 394
input_layer0: layer_id=393: KerasTensor(type_spec=TensorSpec(shape=(1, 25200, 85), dtype=tf.float32, name=None), name='tf.concat_16/concat:0', description="created by layer 'tf.concat_16'")
tf_layers_dict: KerasTensor(type_spec=TensorSpec(shape=(1, 25200, 85), dtype=tf.float32, name=None), name='tf.identity/Identity:0', description="created by layer 'tf.identity'")
====================================================================================
TensorFlow/Keras model building process complete!
saved_model output started ==========================================================
WARNING:absl:Found untraced functions such as _jit_compiled_convolution_op, _jit_compiled_convolution_op, _jit_compiled_convolution_op, _jit_compiled_convolution_op, _jit_compiled_convolution_op while saving (showing 5 of 60). These functions will not be directly callable after loading.
saved_model output complete!
.pb output started ==================================================================
.pb output complete! - saved_model/model_float32.pb
tflite Float32 convertion started ===================================================
WARNING:absl:Found untraced functions such as _jit_compiled_convolution_op, _jit_compiled_convolution_op, _jit_compiled_convolution_op, _jit_compiled_convolution_op, _jit_compiled_convolution_op while saving (showing 5 of 60). These functions will not be directly callable after loading.
Estimated count of arithmetic ops: 5.393 G ops, equivalently 2.697 G MACs
tflite Float32 convertion complete! - saved_model/model_float32.tflite
All the conversion process is finished! =============================================
All issues of YOLOv5 asked the same question. https://github.com/PINTO0309/openvino2tensorflow/issues/115 https://github.com/PINTO0309/openvino2tensorflow/issues/102 https://github.com/PINTO0309/openvino2tensorflow/issues/99 https://github.com/PINTO0309/openvino2tensorflow/issues/91 https://github.com/PINTO0309/openvino2tensorflow/issues/88 https://github.com/PINTO0309/openvino2tensorflow/issues/86 https://github.com/PINTO0309/openvino2tensorflow/issues/82 https://github.com/PINTO0309/openvino2tensorflow/issues/77 https://github.com/PINTO0309/openvino2tensorflow/issues/54 https://github.com/PINTO0309/openvino2tensorflow/issues/50 https://github.com/PINTO0309/openvino2tensorflow/issues/48 https://github.com/PINTO0309/openvino2tensorflow/issues/41 https://github.com/PINTO0309/openvino2tensorflow/issues/38 https://github.com/PINTO0309/openvino2tensorflow/issues/33 https://github.com/PINTO0309/openvino2tensorflow/issues/21
Btw, If you want to rename an input/output layer, you can easily rewrite it manually using this tool.
https://github.com/PINTO0309/tflite2json2tflite
@PINTO0309 super interesting, thanks for the detailed response!
The 3 transpose ops are occurring 1 in each of the output grids P3/8, P4/16, P5/32. The permute op is undesirable as it leaves the output not contiguous, so it requires a .contiguous() method afterward.
I'll revisit the idea of removing the permutation completely from YOLOv5. It's a legacy from YOLOv3 onward, but maybe there's a better way I can restructure this module.
Closed due to no activity for over a week.
https://github.com/PINTO0309/onnx2tf
Straight conversion sample
YOLOv7-tiny with Post-Process (NMS) ONNX to TFLite Float32
Issue Type
Other
OS
Ubuntu
OS architecture
x86_64
Programming Language
Python
Framework
OpenVINO, PyTorch
Download URL for ONNX / OpenVINO IR
yolov5n_openvino_model.zip
Convert Script
Description
In Google Colab I exported YOLOv5n model from PyTorch to OpenVINO, then used openvino2tensorflow command above which produced error message below.
Relevant Log Output
Source code for simple inference testing code
Using https://colab.research.google.com/github/ultralytics/yolov5/blob/master/tutorial.ipynb