edge TPU compiler not mapping any ops to TPU

[busyyang]

I am trying to run a MTCNN model on edge TPU. I have .h5 models and convert to CPU .tflite model with python API:

import tensorflow as tf
import argparse

def keras_to_tflite(keras_model, output_dir, des_model_name):
    converter = tf.lite.TFLiteConverter.from_keras_model_file(keras_model)
    tflite_model = converter.convert()
    open(os.path.join(output_dir, des_model_name), "wb").write(tflite_model)

parser = argparse.ArgumentParser()
parser.add_argument('src_model_name', type=str, help='path and name of .h5 file')
parser.add_argument('des_model_name', type=str, help='name of converted model file')
args = parser.parse_args()
keras_to_tflite(args.src_model_name, output_dir, args.des_model_name)

Then I have the .tflite models. Further more, I use the edge tpu compiler from Google Colab to convert CPU .tflite model to TPU .tflite model.

I convert it successfully and _edgetpu.tflite model generated. But there is no parameter On-chip and all operations are on CPU not Edge TPU. There are just normal operations like Conv2D, MaxPool2D and so on. They should be processed on Edge TPU in my thought.

Edge TPU Compiler version 16.0.384591198
Started a compilation timeout timer of 180 seconds.

Model compiled successfully in 6 ms.

Input model: onet.tflite
Input size: 1.49MiB
Output model: onet_edgetpu.tflite
Output size: 1.49MiB
On-chip memory used for caching model parameters: 0.00B
On-chip memory remaining for caching model parameters: 0.00B
Off-chip memory used for streaming uncached model parameters: 0.00B
Number of Edge TPU subgraphs: 0
Total number of operations: 18
Operation log: onet_edgetpu.log

Model successfully compiled but not all operations are supported by the Edge TPU. A percentage of the model will instead run on the CPU, which is slower. If possible, consider updating your model to use only operations supported by the Edge TPU. For details, visit g.co/coral/model-reqs.
Number of operations that will run on Edge TPU: 0
Number of operations that will run on CPU: 18
See the operation log file for individual operation details.
Compilation child process completed within timeout period.
Compilation succeeded! 

I have upload all the model files on drive

[manoj7410]

@busyyang The very first operator 'Conv2D' itself is working on float32 and cannot be mapped to the TPU. Please perform the quantization on this model and then try to recompile. For more details on quantization please see : https://coral.ai/docs/edgetpu/models-intro/#quantization

[hjonnala]

@busyyang are you able to quantize the model and compile it with edgetpu compiler?

[busyyang]

@busyyang are you able to quantize the model and compile it with edgetpu compiler?

@hjonnala Thanks, I did quantize the model by tf.Lite.Converter and compile it with web-base edgetpu compiler. It works good right now.

Well, may I have another question? I fond that there are postprocess step if I download the trained model from https://coral.ai/models/. In my custom project, how can I add the postprocess into the _edgetpu.tflite instead of do it by post-process code?

[hjonnala]

Hi, There are two ways to create the models that are compatible with Edgeptu.

1. Post integer quantization.
2. Quantization Aware Training

You can use quantization-aware models to avoid post processing.

Please refer to these tutorials for examples.

[busyyang]

Thanks, it help me a lot.