PINTO0309
commented
2 years ago Thank you for posting this thoughtful issue. You have made my investigation go very smoothly.
Some of the conversions had bugs, which have been fixed. https://github.com/PINTO0309/openvino2tensorflow/releases/tag/v1.26.1
There seems to be a slightly larger error, but even after looking at the entire structure of the model, I could not identify which part of the model had the problem.
If possible, try using the model cutting function in the link below to see which layers have larger errors. https://github.com/PINTO0309/openvino2tensorflow#6-10-ability-to-specify-an-output-layer-for-debugging-the-output-values-of-the-model
- Converted model saved_model.zip
One area to pay special attention to during the transformation is whether the shape of the input tensor in Reshape is consistent with the shape of the output tensor after Reshape.
- replace.json
{ "format_version": 2, "layers": [ { "layer_id": "314", "type": "Transpose", "replace_mode": "insert_after", "values": [ 0, 3, 1, 2 ] }, { "layer_id": "315", "type": "Const", "replace_mode": "direct", "values": [ 1, 3, 6, 4, 4 ] }, { "layer_id": "323", "type": "Transpose", "replace_mode": "insert_after", "values": [ 0, 3, 1, 2 ] }, { "layer_id": "324", "type": "Const", "replace_mode": "direct", "values": [ 1, 3, 6, 8, 8 ] }, { "layer_id": "332", "type": "Transpose", "replace_mode": "insert_after", "values": [ 0, 3, 1, 2 ] }, { "layer_id": "333", "type": "Const", "replace_mode": "direct", "values": [ 1, 3, 6, 16, 16 ] } ] }docker run --gpus all -it --rm \ -v `pwd`:/home/user/workdir \ ghcr.io/pinto0309/openvino2tensorflow:latest
$INTEL_OPENVINO_DIR/deployment_tools/model_optimizer/mo.py \ --input_model shuffleNet_4d.onnx \ --data_type FP32 \ --output_dir saved_model/openvino/FP32
openvino2tensorflow \ --model_path saved_model/openvino/FP32/shuffleNet_4d.xml \ --output_saved_model \ --output_pb \ --output_no_quant_float32_tflite \ --weight_replacement_config replace.json
```python
import onnxruntime
import tensorflow as tf
import time
import numpy as np
from pprint import pprint
H=128
W=128
############################################################
onnx_session = onnxruntime.InferenceSession(f'shuffleNet_4d.onnx')
input_name = onnx_session.get_inputs()[0].name
output_name = onnx_session.get_outputs()[0].name
roop = 1
e = 0.0
result = None
inp = np.ones((1,1,H,W), dtype=np.float32)
for _ in range(roop):
s = time.time()
result = onnx_session.run(
[output_name],
{input_name: inp}
)
e += (time.time() - s)
print('ONNX output @@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@')
print(f'elapsed time: {e/roop*1000}ms')
print(f'shape: {result[0].shape}')
pprint(result)
############################################################
interpreter = tf.lite.Interpreter(model_path=f'saved_model/model_float32.tflite', num_threads=4)
interpreter.allocate_tensors()
input_details = interpreter.get_input_details()
output_details = interpreter.get_output_details()
roop = 1
e = 0.0
result = None
inp = np.ones((1,H,W,1), dtype=np.float32)
for _ in range(roop):
s = time.time()
interpreter.set_tensor(input_details[0]['index'], inp)
interpreter.invoke()
result = interpreter.get_tensor(output_details[1]['index'])
e += (time.time() - s)
print('tflite output @@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@')
print(f'elapsed time: {e/roop*1000}ms')
print(f'shape: {result.shape}')
pprint(result)
ONNX output @@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@
elapsed time: 0.5400180816650391ms
shape: (1, 1008, 6)
[array([[[5.26182938e+00, 4.99601841e+00, 1.22561550e+01, 1.36645145e+01,
9.45538282e-04, 9.85830545e-01],
[1.34348326e+01, 5.08123970e+00, 1.37564259e+01, 1.36815252e+01,
4.67479229e-04, 9.85731304e-01],
[2.13319321e+01, 5.01298046e+00, 1.44537868e+01, 1.41183548e+01,
4.15980816e-04, 9.85241652e-01],
...,
[5.63437881e+01, 1.15099594e+02, 1.21474190e+02, 1.02216240e+02,
6.14345074e-04, 9.69812930e-01],
[8.74896545e+01, 1.16726608e+02, 1.10064934e+02, 8.91103439e+01,
5.11735678e-04, 9.71362650e-01],
[1.19493690e+02, 1.12301132e+02, 1.62882797e+02, 1.17599602e+02,
5.25712967e-04, 9.74491775e-01]]], dtype=float32)]
INFO: Created TensorFlow Lite XNNPACK delegate for CPU.
tflite output @@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@
elapsed time: 0.7603168487548828ms
shape: (1, 1008, 6)
array([[[5.28040695e+00, 5.00483131e+00, 1.22718182e+01, 1.36867762e+01,
9.46503831e-04, 9.85802889e-01],
[1.34492149e+01, 5.09168625e+00, 1.37784395e+01, 1.37132292e+01,
4.68593993e-04, 9.85692441e-01],
[2.13420620e+01, 5.02886963e+00, 1.44927731e+01, 1.41707439e+01,
4.17113508e-04, 9.85178471e-01],
...,
[5.66155014e+01, 1.14571739e+02, 1.16065849e+02, 9.84796066e+01,
6.06960442e-04, 9.69192386e-01],
[8.76312714e+01, 1.16028381e+02, 1.04266632e+02, 8.57446976e+01,
5.12930972e-04, 9.70427155e-01],
[1.19529739e+02, 1.11679428e+02, 1.58085449e+02, 1.16086975e+02,
5.34444698e-04, 9.73940670e-01]]], dtype=float32)
beekeeper23
Issue Type
Support
OS
Windows
OS architecture
x86_64
Programming Language
Python
Framework
OpenVINO, PyTorch, ONNX, TensorFlow, TensorFlowLite
Download URL for ONNX / OpenVINO IR
https://drive.google.com/file/d/1gxdHSdOOo4oJWmrFkkBpXUyhVAeqkW2b/view?usp=sharing
Description
I tried to convert the yolov5 algorithm with ShuffleNet from https://github.com/ppogg/YOLOv5-Lite to tflite using the openvino2tensorflow, but the received TensorFlow model had different results on inference, compared to the onnx model.
Also, it is important to note that I used only 4d vectors in the post-processing part of the algorithm.
Below the JSON file and output of inference testing are provided.
Relevant Log Output
Source code for simple inference testing code