Issue with REVERSE_V2 at runtime

[Wirg]

Description

Hello,

Thanks to the coral team for the edge tpus and the support.

I am trying to convert a model to tflite then to compile it for edge_tpu. I am having issue with converting the reverse operation. Is there any alternative ? Shouldn't both operation fail if they can not be run ?

Here is my tflite conversion configuration :

converter.optimizations = [tf.lite.Optimize.DEFAULT]
converter.target_spec.supported_ops = [
    tf.lite.OpsSet.TFLITE_BUILTINS_INT8,
    tf.lite.OpsSet.TFLITE_BUILTINS,
]

converter.target_spec.supported_types = [tf.int8]
converter.inference_input_type = tf.uint8
converter.experimental_new_converter = True

Both operations succeed. But I can not run any of the models with tflite. Logs for the normal version (not tpu) :

ValueError: Didn't find op for builtin opcode 'REVERSE_V2' version '3'
Registration failed.

Logs for the edge tpu version :

RuntimeError: Type 'INT8' is not supported by reverse.Node number 4 (REVERSE_V2) failed to prepare.

Code to run :

from tflite_runtime import interpreter as tflite
import numpy as np
interpreter = tflite.Interpreter("outputs/quantized_size_224.tflite")
interpreter.allocate_tensors()

input_details = interpreter.get_input_details()
output_details = interpreter.get_output_details()

input_shape = input_details[0]['shape']
input_data = np.random.randint(0, 255, input_shape, dtype=np.uint8)
interpreter.set_tensor(input_details[0]['index'], input_data)

interpreter.invoke()

output_data = interpreter.get_tensor(output_details[0]['index'])
print(output_data)

Corresponding weights. outputs.zip

Issue Type

Bug

Operating System

Ubuntu

Coral Device

USB Accelerator

Other Devices

No response

Programming Language

Python 3.7

Relevant Log Output

`$ edgetpu_compiler -s -m 13 quantized_size_224.tflite`
Logs mention that reverse is not working.

Edge TPU Compiler version 16.0.384591198
Input: quantized_size_224.tflite
Output: quantized_size_224_edgetpu.tflite

Operator                       Count      Status

SOFTMAX                        1          Mapped to Edge TPU
STRIDED_SLICE                  3          Operation is working on an unsupported data type
STRIDED_SLICE                  2          More than one subgraph is not supported
STRIDED_SLICE                  4          Mapped to Edge TPU
ADD                            2          Mapped to Edge TPU
ADD                            2          More than one subgraph is not supported
SUB                            1          More than one subgraph is not supported
REVERSE_V2                     2          Operation not supported
NON_MAX_SUPPRESSION_V4         1          Operation is working on an unsupported data type
ARG_MAX                        1          Operation is working on an unsupported data type
QUANTIZE                       1          Operation is otherwise supported, but not mapped due to some unspecified limitation
QUANTIZE                       6          Mapped to Edge TPU
DEQUANTIZE                     6          Operation is working on an unsupported data type
EXP                            1          Operation is working on an unsupported data type
CONCATENATION                  1          More than one subgraph is not supported
CONCATENATION                  3          Mapped to Edge TPU
MAX_POOL_2D                    3          Mapped to Edge TPU
TANH                           5          Mapped to Edge TPU
RESHAPE                        1          Operation is working on an unsupported data type
RESHAPE                        15         Mapped to Edge TPU
CONV_2D                        36         Mapped to Edge TPU
MUL                            4          More than one subgraph is not supported
RESIZE_BILINEAR                3          Mapped to Edge TPU
REDUCE_MAX                     1          Mapped to Edge TPU

[hjonnala]

@wrig I am able to run the inference for both CPU tflite and edgetpu models. Are you using tflite-runtime==2.5.0.post1?

[Wirg]

I am currently running 2.5.0. Should I update ?

[hjonnala]

yes, please get the latest pycoral, tflite runtime and edgetpu runtime from software page

python3 -c "import pycoral.utils.edgetpu; print(pycoral.utils.edgetpu.get_runtime_version())"

output should be below:

BuildLabel(COMPILER=6.3.0 20170516,DATE=redacted,TIME=redacted), RuntimeVersion(14)

numpy==1.21.2
Pillow==8.3.2
pycoral==2.0.0
tflite-runtime==2.5.0.post1

[Wirg]

@hjonnala

So it works for me on cpu. I will test on tpu tomorrow.

I guess the issue has been resolved in the post release then ?

[hjonnala]

@Wirg Are you able to test the tpu model?

[Wirg]

Are you able to test the tpu model?

Hi @hjonnala thanks for contacting me again.

This works "properly" when updating to 2.5.0.post1 and the runtime 14. By properly I mean it performs in the same way than the normal quantized model. Right now, quantization yields really bad performance with my last operations, but I think I have to go and ask tflite team.

I think I have one last question for you : I am compiling using this command edgetpu_compiler -s -m 13 quantized_size_224.tflite .

I would expect :

• either the resulting edgetpu file to properly run with the runtime 13 which is not the case here
• or the compilation to fail which is not the case here

Am I wrong in expecting this ?

[hjonnala]

@wrig if you compile the model with-m 13 flagyou would be able to run this model with edgetpu runtime 13 installation from software page.

This model would work with edgetpu runtime 13and tflite runtime 2.5.0 post1. Some models compiled with Edge TPU Compiler version 16.0.384591198 might require tflite runtime 2.5.0 post1