Output results are not right after converting

[zhongwenkun886]

I want to convert the openvino model (head-pose-estimation-adas-0001) to TFlite, after converting, I use the tensorflow2.4.1 to infer but it get the wrong results compared with the openVINO's output results.

I use the Docker, and the converting command is below:

openvino2tensorflow \
  --model_path head-pose-estimation-adas-0001.xml \
  --output_saved_model \
  --output_pb \
  --output_weight_quant_tflite \
  --output_float16_quant_tflite \
  --output_no_quant_float32_tflite

after converting, the infering code is below:

import numpy as np
import cv2
from tensorflow.lite.python.interpreter import Interpreter

interpreter = Interpreter(model_path='head-pose-estimation.tflite', num_threads=4)
interpreter.allocate_tensors()
input_blob = interpreter.get_input_details()
output_blob = interpreter.get_output_details()

img = cv2.imread("D:\\face.bmp")
img = cv2.resize(img, (60, 60))
img = img.astype(np.float32)
img = img[np.newaxis, :, :, :]

interpreter.set_tensor(input_blob[0]['index'], img)
interpreter.invoke()

out0 = interpreter.get_tensor(output_blob[0]['index'])
out1 = interpreter.get_tensor(output_blob[1]['index'])
out2 = interpreter.get_tensor(output_blob[2]['index'])

The out0,out1 and out2 is the output results, but it is not right compared with the openVINO's output results.

The model head-pose-estimation-adas-0001 link: https://github.com/openvinotoolkit/open_model_zoo/tree/master/models/intel/head-pose-estimation-adas-0001

Thanks very much!

[PINTO0309]

If possible, please describe all the results, including how they differ. I am on my own and have to deal with several issues a day, so I don't have time to verify everything.

[zhongwenkun886]

If possible, please describe all the results, including how they differ. I am on my own and have to deal with several issues a day, so I don't have time to verify everything.

Thank you for replying。

The openVINO(openvino_2021.4.582) infering code is below:

import numpy as np
import cv2
import os
from openvino.inference_engine import IECore

model_xml = "./head-pose-estimation-adas-0001.xml"
model_bin = os.path.splitext(model_xml)[0] + ".bin"
ie = IECore()
net = ie.read_network(model=model_xml, weights=model_bin)
input_blob = next(iter(net.inputs))
exec_net = ie.load_network(network=net, device_name='CPU')

frame = cv2.imread("face.bmp")
img = cv2.resize(frame, (60, 60))
img = img[np.newaxis, :, :, :]  # Batch size axis add
img = img.transpose((0, 3, 1, 2))  # NHWC to NCHW

outputs = exec_net.infer(inputs={input_blob: img})
angle_p, angle_r, angle_y = outputs['angle_p_fc'], outputs['angle_r_fc'], outputs['angle_y_fc']

############ The output results is angle_p=5.79, angle_r=-8.66, angle_y=29.49. But for TFlite，the output results is out0=-2.16, out1=-7.99, out2=-7.84

[PINTO0309]

I have identified that the bug occurs in the last MatMul, and the conversion seems to work fine before MatMul.

[zhongwenkun886]

I have identified that the bug occurs in the last MatMul, and the conversion seems to work fine before MatMul.

So you means that the FC layer of this openvino model can not be convert to TFlite correctly ? And how to fix this bug?

[PINTO0309]

We need to do something about the shape of the Reshape immediately before. I'm working on it.

There is a big difference in the results between the flow of [1,2,2,128]->Reshape[1,512]->MatMul[1,512] and the flow of [1,128,2,2]->Reshape[1,512]->MatMul[1,512].

Therefore, the MatMul transformation is normal. This probably requires transposition by Transpose just before Reshape.

[PINTO0309]

The result of forcing Transpose to be inserted just before Reshape. All results are now correct.

• tflite results

  fc_p: -6.788656711578369
  fc_r: 9.602302551269531
  fc_y: 7.484607696533203

• openvino results

  {'angle_p_fc': array([[-6.788691]], dtype=float32),
  'angle_r_fc': array([[9.602323]], dtype=float32),
  'angle_y_fc': array([[7.4845986]], dtype=float32)}

• test code

  
  import numpy as np
  import cv2
  from tensorflow.lite.python.interpreter import Interpreter
  from pprint import pprint

interpreter = Interpreter(model_path='saved_model/model_float32.tflite', num_threads=4) interpreter.allocate_tensors() input_blob = interpreter.get_input_details() output_blob = interpreter.get_output_details()

""" [{'dtype': <class 'numpy.float32'>, 'index': 0, 'name': 'data', 'quantization': (0.0, 0), 'quantization_parameters': {'quantized_dimension': 0, 'scales': array([], dtype=float32), 'zero_points': array([], dtype=int32)}, 'shape': array([ 1, 60, 60, 3], dtype=int32), 'shape_signature': array([ 1, 60, 60, 3], dtype=int32), 'sparsity_parameters': {}}] """ pprint(input_blob) pprint(output_blob)

img = cv2.imread("test.jpg") img = cv2.resize(img, (60, 60)) img = img.astype(np.float32) img = img[np.newaxis, :, :, :]

interpreter.set_tensor(input_blob[0]['index'], img) interpreter.invoke()

out0 = interpreter.get_tensor(output_blob[0]['index']) out1 = interpreter.get_tensor(output_blob[1]['index']) out2 = interpreter.get_tensor(output_blob[2]['index'])

""" [{'dtype': <class 'numpy.float32'>, 'index': 85, 'name': 'Identity', 'quantization': (0.0, 0), 'quantization_parameters': {'quantized_dimension': 0, 'scales': array([], dtype=float32), 'zero_points': array([], dtype=int32)}, 'shape': array([1, 1], dtype=int32), 'shape_signature': array([1, 1], dtype=int32), 'sparsity_parameters': {}}, {'dtype': <class 'numpy.float32'>, 'index': 88, 'name': 'Identity_1', 'quantization': (0.0, 0), 'quantization_parameters': {'quantized_dimension': 0, 'scales': array([], dtype=float32), 'zero_points': array([], dtype=int32)}, 'shape': array([1, 1], dtype=int32), 'shape_signature': array([1, 1], dtype=int32), 'sparsity_parameters': {}}, {'dtype': <class 'numpy.float32'>, 'index': 80, 'name': 'Identity_2', 'quantization': (0.0, 0), 'quantization_parameters': {'quantized_dimension': 0, 'scales': array([], dtype=float32), 'zero_points': array([], dtype=int32)}, 'shape': array([1, 1], dtype=int32), 'shape_signature': array([1, 1], dtype=int32), 'sparsity_parameters': {}}] """ print(f'fc_p: {np.sum(out2)}') print(f'fc_r: {np.sum(out1)}') print(f'fc_y: {np.sum(out0)}')

from openvino.inference_engine import IENetwork, IECore

ie = IECore() model='head-pose-estimation-adas-0001' net = ie.read_network(f'{model}.xml', f'{model}.bin') input_blob = next(iter(net.input_info)) out_blob = next(iter(net.outputs)) exec_net = ie.load_network(net, 'CPU')

""" 'data' """ pprint(input_blob) """ {'angle_p_fc': <openvino.inference_engine.ie_api.DataPtr object at 0x7fa6e5eb9530>, 'angle_r_fc': <openvino.inference_engine.ie_api.DataPtr object at 0x7fa6e32c74b0>, 'angle_y_fc': <openvino.inference_engine.ie_api.DataPtr object at 0x7fa6e32c7430>} """ pprint(net.outputs)

res = exec_net.infer(inputs={input_blob: img.transpose(0,3,1,2)}) pprint(res)

[PINTO0309]

I have committed the converted model. https://github.com/PINTO0309/PINTO_model_zoo/tree/main/134_head-pose-estimation-adas-0001

[zhongwenkun886]

Thank you very much!

[MekhailS]

@PINTO0309 Could you please share .json config used for forcing inserting Transpose for this model? The config would much help to understand which values are needed in Transpose operation (values in OpenVino format or Tensorflow's)

[PINTO0309]

It is bad manners to continue a discussion on a closed issue. Why don't you try to read the README?

https://github.com/PINTO0309/openvino2tensorflow#6-7-replace-weights-or-constant-values-in-const-op-and-add-transpose-or-reshape-just-before-the-operation-specified-by-layer_id