search

PINTO0309 / openvino2tensorflow

This script converts the ONNX/OpenVINO IR model to Tensorflow's saved_model, tflite, h5, tfjs, tftrt(TensorRT), CoreML, EdgeTPU, ONNX and pb. PyTorch (NCHW) -> ONNX (NCHW) -> OpenVINO (NCHW) -> openvino2tensorflow -> Tensorflow/Keras (NHWC/NCHW) -> TFLite (NHWC/NCHW). And the conversion from .pb to saved_model and from saved_model to .pb and from .pb to .tflite and saved_model to .tflite and saved_model to onnx. Support for building environments with Docker. It is possible to directly access the host PC GUI and the camera to verify the operation. NVIDIA GPU (dGPU) support. Intel iHD GPU (iGPU) support.
MIT License
334 stars 40 forks source link

openvino int8/fp32 to tflite convertion #103

Closed danilopau closed 2 years ago

danilopau commented 2 years ago

Issue Type

Bug

OS

Ubuntu

OS architecture

x86_64

Programming Language

Python

Framework

OpenVINO

Download URL for ONNX / OpenVINO IR

omz_downloader --name unet-camvid-onnx-0001 then pick the FP16-INT8 version unet-camvid-onnx-0001-xml and unet-camvid-onnx-0001.bin

Convert Script

(openvino2tf) rivendell@rivendell-HP-Z820-Workstation:~/openvino2tf/openvino2tensorflow/openvino2tensorflow$ python3 openvino2tensorflow.py --model_path /home/rivendell/openvino_zoo/unet-camvid-onnx-0001/intel/unet-camvid-onnx-0001/FP16-INT8/unet-camvid-onnx-0001.xml --output_no_quant_float32_tflite
TensorFlow/Keras model building process starts ======================================
Layer structure =====================================================================
layer_type: Input
layer_id: 12
tf_layers_dict: KerasTensor(type_spec=TensorSpec(shape=(1, 368, 480, 3), dtype=tf.float32, name='input_1'), name='input_1', description="created by layer 'input_1'")
====================================================================================
layer_type: Const
layer_id: 0
tf_layers_dict_shape: (1,)
tf_layers_dict_value: [-1.3650802]
====================================================================================
layer_type: Const
layer_id: 1
tf_layers_dict_shape: (1,)
tf_layers_dict_value: [7.9941273]
====================================================================================
layer_type: Const
layer_id: 2
tf_layers_dict_shape: (1,)
tf_layers_dict_value: [-1.3650802]
====================================================================================
layer_type: Const
layer_id: 3
tf_layers_dict_shape: (1,)
tf_layers_dict_value: [7.9941273]
====================================================================================
layer_type: Const
layer_id: 4
tf_layers_dict_shape: (1,)
tf_layers_dict_value: [0.]
====================================================================================
layer_type: Const
layer_id: 5
tf_layers_dict_shape: (1,)
tf_layers_dict_value: [10.761472]
====================================================================================
layer_type: Const
layer_id: 6
tf_layers_dict_shape: (1,)
tf_layers_dict_value: [0.]
====================================================================================
layer_type: Const
layer_id: 7
tf_layers_dict_shape: (1,)
tf_layers_dict_value: [10.761472]
====================================================================================
layer_type: Const
layer_id: 8
tf_layers_dict_shape: (1,)
tf_layers_dict_value: [-156.59842]
====================================================================================
layer_type: Const
layer_id: 9
tf_layers_dict_shape: (1,)
tf_layers_dict_value: [155.375]
====================================================================================
layer_type: Const
layer_id: 10
tf_layers_dict_shape: (1,)
tf_layers_dict_value: [-156.59842]
====================================================================================
layer_type: Const
layer_id: 11
tf_layers_dict_shape: (1,)
tf_layers_dict_value: [155.375]
====================================================================================
layer_type: Const
layer_id: 13
tf_layers_dict_shape: (1, 3, 1, 1)
tf_layers_dict_value: [[[[105.6875]]

  [[103.3125]]

  [[ 99.625 ]]]]
====================================================================================
layer_type: Convert
layer_id: 14
input_layer0_shape: layer_id=13: Const(ndarray).shape (1, 3, 1, 1)
input_layer0_value: layer_id=13: [[[[105.6875]]

  [[103.3125]]

  [[ 99.625 ]]]]
tf_layers_dict_shape: (1, 3, 1, 1)
tf_layers_dict: [[[[105.6875]]

  [[103.3125]]

  [[ 99.625 ]]]]
====================================================================================
ERROR: Exception encountered when calling layer "tf.math.subtract" (type TFOpLambda).

Dimensions must be equal, but are 368 and 3 for '{{node tf.math.subtract/Sub}} = Sub[T=DT_FLOAT](Placeholder, tf.math.subtract/Sub/y)' with input shapes: [1,368,480,3], [1,3,1,1].

Call arguments received:
  • x=tf.Tensor(shape=(1, 368, 480, 3), dtype=float32)
  • y=tf.Tensor(shape=(1, 3, 1, 1), dtype=float32)
  • name=None
ERROR: model_path  : /home/rivendell/openvino_zoo/unet-camvid-onnx-0001/intel/unet-camvid-onnx-0001/FP16-INT8/unet-camvid-onnx-0001.xml
ERROR: weights_path: /home/rivendell/openvino_zoo/unet-camvid-onnx-0001/intel/unet-camvid-onnx-0001/FP16-INT8/unet-camvid-onnx-0001.bin
ERROR: layer_id    : 15
ERROR: input_layer0 layer_id=12: KerasTensor(type_spec=TensorSpec(shape=(1, 368, 480, 3), dtype=tf.float32, name='input_1'), name='input_1', description="created by layer 'input_1'")
ERROR: input_layer1 layer_id=14: tf.Tensor(
[[[[105.6875]]

  [[103.3125]]

  [[ 99.625 ]]]], shape=(1, 3, 1, 1), dtype=float32)
ERROR: The trace log is below.
Traceback (most recent call last):
  File "openvino2tensorflow.py", line 4725, in convert
    inp = tf.math.subtract(x, y)
  File "/home/rivendell/.local/lib/python3.8/site-packages/tensorflow/python/util/traceback_utils.py", line 153, in error_handler
    raise e.with_traceback(filtered_tb) from None
  File "/home/rivendell/.local/lib/python3.8/site-packages/keras/layers/core/tf_op_layer.py", line 107, in handle
    return TFOpLambda(op)(*args, **kwargs)
  File "/home/rivendell/.local/lib/python3.8/site-packages/keras/utils/traceback_utils.py", line 67, in error_handler
    raise e.with_traceback(filtered_tb) from None
ValueError: Exception encountered when calling layer "tf.math.subtract" (type TFOpLambda).

Dimensions must be equal, but are 368 and 3 for '{{node tf.math.subtract/Sub}} = Sub[T=DT_FLOAT](Placeholder, tf.math.subtract/Sub/y)' with input shapes: [1,368,480,3], [1,3,1,1].

Call arguments received:
  • x=tf.Tensor(shape=(1, 368, 480, 3), dtype=float32)
  • y=tf.Tensor(shape=(1, 3, 1, 1), dtype=float32)
  • name=None
ERROR: Please refer to 6-7 in the README first. https://github.com/PINTO0309/openvino2tensorflow
(openvino2tf) rivendell@rivendell-HP-Z820-Workstation:~/openvino2tf/openvino2tensorflow/openvino2tensorflow$

Description

need to convert openvino to tflite with int8 and/or fp32 and without fp16

Relevant Log Output

see above

Source code for simple inference testing code

No response

PINTO0309 commented 2 years ago

Fix: https://github.com/PINTO0309/openvino2tensorflow/commit/f719975560abedfe9beabfd371ce60e24cda043a v1.29.6

wget https://storage.openvinotoolkit.org/repositories/open_model_zoo/2022.1/models_bin/3/unet-camvid-onnx-0001/FP32/unet-camvid-onnx-0001.xml
wget https://storage.openvinotoolkit.org/repositories/open_model_zoo/2022.1/models_bin/3/unet-camvid-onnx-0001/FP32/unet-camvid-onnx-0001.bin

docker run -it --rm \
-v `pwd`:/home/user/workdir \
ghcr.io/pinto0309/openvino2tensorflow:latest

pip install openvino2tensorflow --upgrade

openvino2tensorflow \
--model_path unet-camvid-onnx-0001.xml \
--output_saved_model \
--output_pb \
--non_verbose \
--output_no_quant_float32_tflite

TensorFlow/Keras model building process complete!
saved_model output started ==========================================================
saved_model output complete!
.pb output started ==================================================================
.pb output complete! - saved_model/model_float32.pb
tflite Float32 convertion started ===================================================
tflite Float32 convertion complete! - saved_model/model_float32.tflite
All the conversion process is finished! =============================================