YOLACT conversion to EdgeTPU

[awur978]

Issue Type

Others

OS

Ubuntu

OS architecture

x86_64

Programming Language

Python

Framework

OpenVINO

Download URL for ONNX / OpenVINO IR

https://drive.google.com/file/d/1_yelX7TPZclzW6U0DAmSKdmmUZO5VyEr/view?usp=sharing

Convert Script

I installed the openvino2tensorflow using docker available here: https://github.com/PINTO0309/openvino2tensorflow/blob/main/README.md

user@b13076afa5dc:~/workdir$ openvino2tensorflow   --model_path yolact_regnetx_800MF_1970_800000.simplified.xml   --output_saved_model   --output_pb   --output_weight_quant_tflite   --output_float16_quant_tflite  --output_integer_quant_tflite --output_full_integer_quant_tflite

Description

I am trying to convert YOLACT to edge TPU, the PINTO0309 model zoo stated that YOLACT was successfully converted to edgeTPU. I have followed your tutorial but whenever I get to the stage of converting from tflite to edgetpu I am unable to.

Relevant Log Output

TensorFlow/Keras model building process starts ======================================
Layer structure =====================================================================
layer_type: Input
layer_id: 0
tf_layers_dict: KerasTensor(type_spec=TensorSpec(shape=(1, 512, 512, 3), dtype=tf.float32, name='input_1'), name='input_1', description="created by layer 'input_1'")
====================================================================================
layer_type: Const
layer_id: 1
tf_layers_dict_shape: (32, 3, 3, 3)

      ...,

        [[-6.21931767e-03,  1.00612426e-02,  3.90599016e-03],
         [ 1.87847152e-04,  1.74555201e-02,  1.99538041e-02],
         [-1.45665687e-02, -1.24538578e-02, -5.01801772e-03]],

        [[ 9.05317720e-03,  4.98088216e-03,  1.61416654e-03],
         [ 9.07150004e-03,  1.37506835e-02,  1.16696525e-02],
         [ 1.43749146e-02,  2.17593201e-02,  8.99904408e-03]],

        [[-4.27000038e-03, -5.52132167e-03,  2.13511102e-03],
         [ 2.46740784e-03,  1.38835600e-02, -3.52095556e-03],
         [ 5.54961793e-04, -8.59608408e-03,  8.56689084e-03]]]],
      dtype=float32)
ERROR: input_layer1 layer_id=366: KerasTensor(type_spec=TensorSpec(shape=(1, 32, 32, 256), dtype=tf.float32, name=None), name='tf.nn.relu_57/Relu:0', description="created by layer 'tf.nn.relu_57'")
ERROR: The trace log is below.

Traceback (most recent call last):
  File "/usr/local/bin/openvino2tensorflow", line 887, in convert
    tf_layers_dict[layer_id] = tf.nn.conv2d(
  File "/usr/local/lib/python3.8/dist-packages/tensorflow/python/util/traceback_utils.py", line 153, in error_handler
    raise e.with_traceback(filtered_tb) from None
  File "/usr/local/lib/python3.8/dist-packages/keras/layers/core/tf_op_layer.py", line 107, in handle
    return TFOpLambda(op)(*args, **kwargs)
  File "/usr/local/lib/python3.8/dist-packages/keras/utils/traceback_utils.py", line 67, in error_handler
    raise e.with_traceback(filtered_tb) from None
ValueError: Exception encountered when calling layer "tf.nn.conv2d" (type TFOpLambda).

Depth of input (3) is not a multiple of input depth of filter (32) for '{{node tf.nn.conv2d/Conv2D}} = Conv2D[T=DT_FLOAT, data_format="NHWC", dilations=[1, 1, 1, 1], explicit_paddings=[], padding="VALID", strides=[1, 1, 1, 1], use_cudnn_on_gpu=true](tf.nn.conv2d/Conv2D/input, Placeholder)' with input shapes: [256,258,5,3], [1,32,32,256].

Call arguments received by layer "tf.nn.conv2d" (type TFOpLambda):
  • input=tf.Tensor(shape=(256, 258, 5, 3), dtype=float32)
  • filters=tf.Tensor(shape=(1, 32, 32, 256), dtype=float32)
  • strides=['1', '1']
  • padding='VALID'
  • data_format=NHWC
  • dilations=['1', '1']
  • name=None
ERROR: Please refer to 6-7 in the README first. https://github.com/PINTO0309/openvino2tensorflow

Source code for simple inference testing code

No response

[PINTO0309]

.xml is broken. Also, the model is too large to be converted to EdgeTPU.

[awur978]

Thank you so much for your prompt response. Can you be so kind to tell me how to fix broken .xml? Did you use the resnet-50 for your yolact backend implementation? if no what did you use. Thank you and sorry if these are silly questions

[PINTO0309]

I do not know what structure the ONNX file you generated is. I can't answer any questions without information.

[awur978]

Thank you for responding. My previous architecture was using the Darknet53-FPN as backbone but based on your suggestion that the model is too large to be converted to EdgeTPU, I used the code from this git repo: https://github.com/Ma-Dan/yolact to generate the ONNX file for Yolact with backbone Resnet50-FPN. I generated the .xml using the openvino-dev package. The ONNX file as well as the .xml is available at this link: https://drive.google.com/file/d/1fGWKceU6oDb9MYTqx3b-ZQ5tal-2YHL3/view?usp=sharing . Am able to convert to tflite successfully but was not successful with the conversion to edgetpu with the same error as before. Please let me know if you need further information. Thank you.

[PINTO0309]

I will say it again. The model is too large to be converted to EdgeTPU.