larrywal-express commented 2 years ago

Issue Type

Others

OS

Windows

OS architecture

x86_64

Programming Language

Python

Framework

PyTorch, TensorFlow

Download URL for ONNX / OpenVINO IR

lite4D.zip lite.zip the attached onnx, xml/bin

Convert Script

lite.zip

Description

I succeeded converting onnx in 4D to xml, but getting error to tflite conversion. The onnx and xml/bin attached including the output error.

Relevant Log Output

layer_type: Const
layer_id: 404
tf_layers_dict: (1, 1, 1024, 2)
====================================================================================
ERROR: Exception encountered when calling layer "tf.math.add_81" (type TFOpLambda).

Dimensions must be equal, but are 3 and 1024 for '{{node tf.math.add_81/Add}} = AddV2[T=DT_FLOAT](Placeholder, tf.math.add_81/Add/y)' with input shapes: [1,3,2,9], [1,1024,2,1].

Call arguments received:
  • x=tf.Tensor(shape=(1, 3, 2, 9), dtype=float32)
  • y=array([[[[ 0.],
         [ 0.]],

        [[ 1.],
         [ 0.]],

        [[ 2.],
         [ 0.]],

        ...,

        [[29.],
         [31.]],

        [[30.],
         [31.]],

        [[31.],
         [31.]]]], dtype=float32)
  • name=None
ERROR: model_path  : lite/lite.xml
ERROR: weights_path: lite/lite.bin
ERROR: layer_id    : 405
ERROR: input_layer0 layer_id=403: KerasTensor(type_spec=TensorSpec(shape=(1, 3, 2, 9), dtype=tf.float32, name=None), name='tf.math.add_78/Add:0', description="created by layer 'tf.math.add_78'")
ERROR: input_layer1 layer_id=404: Const(ndarray).shape  (1, 1, 1024, 2)
array([[[[ 0.,  0.],
         [ 1.,  0.],
         [ 2.,  0.],
         ...,
         [29., 31.],
         [30., 31.],
         [31., 31.]]]], dtype=float32)
ERROR: The trace log is below.
Traceback (most recent call last):
  File "openvino2tensorflow.py", line 994, in convert
    tf_layers_dict[layer_id] = tf.math.add(tf_layers_dict[edge_id0], tf_layers_dict[edge_id1].transpose(0,2,3,1))
  File "D:\Anaconda3\lib\site-packages\tensorflow\python\util\traceback_utils.py", line 153, in error_handler
    raise e.with_traceback(filtered_tb) from None
  File "D:\Anaconda3\lib\site-packages\keras\layers\core\tf_op_layer.py", line 107, in handle
    return TFOpLambda(op)(*args, **kwargs)
  File "D:\Anaconda3\lib\site-packages\keras\utils\traceback_utils.py", line 67, in error_handler
    raise e.with_traceback(filtered_tb) from None
ValueError: Exception encountered when calling layer "tf.math.add_81" (type TFOpLambda).

Dimensions must be equal, but are 3 and 1024 for '{{node tf.math.add_81/Add}} = AddV2[T=DT_FLOAT](Placeholder, tf.math.add_81/Add/y)' with input shapes: [1,3,2,9], [1,1024,2,1].

Call arguments received:
  • x=tf.Tensor(shape=(1, 3, 2, 9), dtype=float32)
  • y=array([[[[ 0.],
         [ 0.]],

        [[ 1.],
         [ 0.]],

        [[ 2.],
         [ 0.]],

        ...,

        [[29.],
         [31.]],

        [[30.],
         [31.]],

        [[31.],
         [31.]]]], dtype=float32)
  • name=None

Source code for simple inference testing code

mo --input_model lite4D.onnx --model_name lite/lite --data_type FP32 python openvino2tensorflow.py --model_path lite/lite.xml --output_saved_model --output_pb --output_no_quant_float32_tflite

PINTO0309 commented 2 years ago

The output of your lite4D.onnx and the converted model_float32.tflite now match perfectly. Incidentally, the situation is exactly the same as with the 5D, and there is almost no error between the ONNX output and the tflite output. If the inference results are different from what you expect, there is a problem with your inference code.

model_float32.tflite.zip

replace.json

{
"format_version": 2,
"layers": [
    {
        "layer_id": "356",
        "type": "Const",
        "replace_mode": "direct",
        "values": [
            1,
            256,
            3,
            9
        ]
    },
    {
        "layer_id": "358",
        "type": "Const",
        "replace_mode": "direct",
        "values": [
            0,
            2,
            1,
            3
        ]
    },
    {
        "layer_id": "391",
        "type": "Const",
        "replace_mode": "direct",
        "values": [
            1,
            1024,
            3,
            9
        ]
    },
    {
        "layer_id": "393",
        "type": "Const",
        "replace_mode": "direct",
        "values": [
            0,
            2,
            1,
            3
        ]
    },
    {
        "layer_id": "397",
        "type": "Const",
        "replace_mode": "direct",
        "values": [
            4
        ]
    },
    {
        "layer_id": "418",
        "type": "Const",
        "replace_mode": "direct",
        "values": [
            4
        ]
    },
    {
        "layer_id": "412",
        "type": "Cast",
        "replace_mode": "insert_after",
        "values": "f32"
    },
    {
        "layer_id": "433",
        "type": "Cast",
        "replace_mode": "insert_after",
        "values": "f32"
    },
    {
        "layer_id": "414",
        "type": "Concat",
        "replace_mode": "change_axis",
        "values": 3
    },
    {
        "layer_id": "435",
        "type": "Concat",
        "replace_mode": "change_axis",
        "values": 3
    }
]
}

convert


docker run -it --rm \
-v `pwd`:/home/user/workdir \
ghcr.io/pinto0309/openvino2tensorflow:latest

$INTEL_OPENVINO_DIR/deployment_tools/model_optimizer/mo.py \ --input_model lite4D.onnx \ --data_type FP32 \ --output_dir openvino/FP32 \ --model_name lite4D

openvino2tensorflow \ --model_path openvino/FP32/lite4D.xml \ --output_saved_model \ --output_pb \ --output_no_quant_float32_tflite \ --non_verbose \ --weight_replacement_config replace.json

- onnx_tflite_test.py
```python
import onnxruntime
import tensorflow as tf
import time
import numpy as np
from pprint import pprint

H=512
W=512
MODEL='model_float32'

############################################################

onnx_session = onnxruntime.InferenceSession('lite4D.onnx')
input_name = onnx_session.get_inputs()[0].name
output_name = onnx_session.get_outputs()[0].name

roop = 1
e = 0.0
result = None
inp = np.ones((1,3,H,W), dtype=np.float32)
for _ in range(roop):
    s = time.time()
    result = onnx_session.run(
        [output_name],
        {input_name: inp}
    )
    e += (time.time() - s)
print('ONNX output @@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@')
print(f'elapsed time: {e/roop*1000}ms')
print(f'shape: {result[0].shape}')
pprint(result)

############################################################

interpreter = tf.lite.Interpreter(model_path=f'saved_model/{MODEL}.tflite', num_threads=4)
interpreter.allocate_tensors()
input_details = interpreter.get_input_details()
output_details = interpreter.get_output_details()

roop = 1
e = 0.0
result = None
inp = np.ones((1,H,W,3), dtype=np.float32)
for _ in range(roop):
    s = time.time()
    interpreter.set_tensor(input_details[0]['index'], inp)
    interpreter.invoke()
    result = interpreter.get_tensor(output_details[1]['index'])
    e += (time.time() - s)
print('tflite output @@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@')
print(f'elapsed time: {e/roop*1000}ms')
print(f'shape: {result.shape}')
pprint(result)

Results
```
$ python onnx_tflite_test.py 
```

ONNX output @@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@ elapsed time: 6.284475326538086ms shape: (1, 3840, 9) [array([[[5.2931595e+00, 9.6953697e+00, 2.4912457e+01, ..., 1.5526780e-01, 2.2650501e-01, 4.0879369e-01], [2.0134949e+01, 6.2680893e+00, 3.3342621e+01, ..., 2.1189997e-01, 2.2379622e-01, 4.7713119e-01], [3.8248707e+01, 1.7845745e+00, 3.6460529e+01, ..., 1.0526398e-01, 1.7205426e-01, 8.1319189e-01], ..., [4.3373840e+02, 4.8147830e+02, 1.3926814e+02, ..., 1.4624476e-02, 1.3875341e-01, 2.2544542e-01], [4.6488284e+02, 4.8298419e+02, 1.3351683e+02, ..., 1.9375145e-02, 2.0011839e-01, 2.2959533e-01], [4.8577332e+02, 4.8264972e+02, 1.1711755e+02, ..., 2.0683646e-02, 2.8468835e-01, 3.3594516e-01]]], dtype=float32)] INFO: Created TensorFlow Lite XNNPACK delegate for CPU. tflite output @@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@ elapsed time: 22.71747589111328ms shape: (1, 3840, 9) array([[[5.29315758e+00, 9.69537735e+00, 2.49124527e+01, ..., 1.55268013e-01, 2.26505280e-01, 4.08792853e-01], [2.01349468e+01, 6.26809692e+00, 3.33426208e+01, ..., 2.11900204e-01, 2.23796397e-01, 4.77130651e-01], [3.82487106e+01, 1.78458500e+00, 3.64605331e+01, ..., 1.05264165e-01, 1.72054380e-01, 8.13191533e-01], ..., [4.33738434e+02, 4.81478302e+02, 1.39268143e+02, ..., 1.46242362e-02, 1.38754994e-01, 2.25445867e-01], [4.64882843e+02, 4.82984192e+02, 1.33516907e+02, ..., 1.93749964e-02, 2.00120524e-01, 2.29595125e-01], [4.85773315e+02, 4.82649719e+02, 1.17117645e+02, ..., 2.06834618e-02, 2.84691095e-01, 3.35944831e-01]]], dtype=float32)



duplicate issue: https://github.com/PINTO0309/openvino2tensorflow/issues/91

larrywal-express commented 2 years ago

@PINTO0309 thanks. the results is the same.

if I want to apply the replace.json how do I read the .bin associated with. xml? Because am still trying to figure out the replacement process.

PINTO0309 commented 2 years ago

stop using Google Colaboratory and use WSL2 + Docker

Run the conversion command with the --non_verbose option removed


docker run -it --rm \
-v `pwd`:/home/user/workdir \
ghcr.io/pinto0309/openvino2tensorflow:latest

$INTEL_OPENVINO_DIR/deployment_tools/model_optimizer/mo.py \ --input_model lite4D.onnx \ --data_type FP32 \ --output_dir openvino/FP32 \ --model_name lite4D

openvino2tensorflow \ --model_path openvino/FP32/lite4D.xml \ --output_saved_model \ --output_pb \ --output_no_quant_float32_tflite \

--non_verbose \

--weight_replacement_config replace.json


3. Almost all values read from the .bin will be displayed in the console.

larrywal-express commented 2 years ago

Thanks

PINTO0309 commented 2 years ago

Since there seems to be no progress, I'll close it.

PINTO0309 / openvino2tensorflow

onnx: xml/bin to tflite conversion error #93

Issue Type

OS

OS architecture

Programming Language

Framework

Download URL for ONNX / OpenVINO IR

Convert Script

Description

Relevant Log Output

Source code for simple inference testing code

--non_verbose \