Process to Convert TF2 SSD MobileNetV* to be ran on USB Accelerator

[zcase]

Description

Currently have trained a custom object detection model using the TF2 object detection api using tensorflow 2.10. I have exported it to a .tflite file and then went to compile it to be used on the USB Accelerator but it fails.

All the tutorials out there are for TF1 or using the EfficientDet for TF2. are there any tutorials or things to watch out for when doing a ssd model using the TF2 object detection api?

Click to expand!

### Issue Type Documentation Feature Request ### Operating System Linux ### Coral Device USB Accelerator ### Other Devices _No response_ ### Programming Language Python 3.8 ### Relevant Log Output _No response_

[hjonnala]

Can you share the steps that you followed so far and the tflite file here please?

[zcase]

@hjonnala

Steps to reproduce:

Download SSD MobileNet V1 FPN 640x640 from TF2 Model Zoo

Extract tar.gz

Export saved model to tflite compatible model (Note there is a bug for 2.10 I have found and suggested a fix for here python models/research/object_detection/export_tflite_graph_tf2.py --pipeline_config_path=PATH_TO_DIR/ssd_mobilenet_v1_fpn_640x640_coco17_tpu-8/pipeline.config --trained_checkpoint_dir=PAT_TO_DIR/ssd_mobilenet_v1_fpn_640x640_coco17_tpu-8/checkpoint/ --output_directory=PATH_TO_DIR/ssd_mobilenet_v1_fpn_640x640_coco17_tpu-8/tflite

Convert tflite saved_model to quantized tflite model using the following code (i called it test.py): python test.py PATH_TO_DIR/ssd_mobilenet_v1_fpn_640x640_coco17_tpu-8/tflite/saved_model/

Grab a test dataset to do object detection and replace the phrase INSERT_PATH_TO_IMG_DIR with the path to the downloaded dataset.

# ORIGINAL CODE: https://colab.research.google.com/drive/1XuWniYM6q_0Uffp6SqSC5GUoS4Bd78RM?usp=sharing#scrollTo=0gV8vr6nN-z9
import cv2
import os
import random
import numpy as np
import tensorflow as tf

print('\n')
print('-'*80)
print(tf.__version__)
print('-'*80)

def _representative_dataset_gen_full():
    # root = 'val2017/'
    # pattern = "*.jpg"
    root = 'INSERT_PATH_TO_IMG_DIR'
    pattern = "*.png"
    imagePaths = []
    for cur_path in os.listdir(root):
        if cur_path.endswith('.png'):
          imagePaths.append(os.path.join(root, cur_path))

    print(f'Found {len(imagePaths)} images!')
    # for path, subdirs, files in os.walk(root):
    #     for name in files:
    #         if fnmatch(name, pattern):
    #             imagePaths.append(root + name)

    for _ in range(10):
      random.shuffle(imagePaths)
    # rep_data = imagePaths[:500]
    rep_data = imagePaths[:50]
    images = []
    for index, image_path in enumerate(rep_data): 
        if index % 50 == 0:
            print(index)
        # start = time.time()
        img = cv2.imread(image_path)
        # original_image = img.copy()
        img = cv2.cvtColor(img, cv2.COLOR_BGR2RGB) 
        img = img.astype(np.float32)
        img = cv2.normalize(img, img, alpha=0, beta=1, norm_type=cv2.NORM_MINMAX, dtype=cv2.CV_32F)
        img = cv2.resize(img, (640, 640))
        img = img.reshape(1, 640, 640, 3)
        # end = time.time()
        # print(f'{index}: took: {(end-start)}s')

        yield [img.astype("float32")]

path_to_saved_model = os.sys.argv[1]
path_to_saved_model_dir = os.path.abspath(os.path.dirname(path_to_saved_model))
_QUANT_TFLITE_MODEL_PATH_FULL_REP = os.path.abspath(os.path.join(path_to_saved_model_dir, '../quant_model.tflite'))
converter = tf.lite.TFLiteConverter.from_saved_model(path_to_saved_model)
converter.optimizations = [tf.lite.Optimize.DEFAULT]
converter.representative_dataset = _representative_dataset_gen_full
converter.target_spec.supported_ops = [tf.lite.OpsSet.TFLITE_BUILTINS_INT8]
converter.inference_input_type = tf.uint8  # or tf.uint8
converter.inference_output_type = tf.uint8  # or tf.uint8

quant_tflite_model = converter.convert()
with open(_QUANT_TFLITE_MODEL_PATH_FULL_REP, 'wb') as f:
  f.write(quant_tflite_model)
print(f'Saved model to: {_QUANT_TFLITE_MODEL_PATH_FULL_REP}')

Then run the edgetpu_compiler on quantized model:

edgetpu_compiler -s -d -a -o PATH_TO_DIR/ssd_mobilenet_v1_fpn_640x640_coco17_tpu-8/tflite/ PATH_TO_DIR/ssd_mobilenet_v1_fpn_640x640_coco17_tpu-8/tflite/quant_model.tflite 

The output is then:

Edge TPU Compiler version 16.0.384591198
Searching for valid delegate with step 1
Try to compile segment with 115 ops
Started a compilation timeout timer of 180 seconds.
Compilation child process completed within timeout period.
Compilation failed! 
Try to compile segment with 114 ops
Intermediate tensors: StatefulPartitionedCall:2,StatefulPartitionedCall:3,StatefulPartitionedCall:0,tfl.quantize3
Started a compilation timeout timer of 180 seconds.
Compilation child process completed within timeout period.
Compilation failed! 
Try to compile segment with 113 ops
Intermediate tensors: StatefulPartitionedCall:0,StatefulPartitionedCall:2,tfl.quantize1,tfl.quantize3
Started a compilation timeout timer of 180 seconds.
Compilation child process completed within timeout period.
Compilation failed! 
Try to compile segment with 112 ops
Intermediate tensors: tfl.quantize2,StatefulPartitionedCall:0,tfl.quantize3,tfl.quantize1
Started a compilation timeout timer of 180 seconds.
Compilation child process completed within timeout period.
Compilation failed! 
Try to compile segment with 111 ops
Intermediate tensors: tfl.quantize4,tfl.quantize2,tfl.quantize3,tfl.quantize1
Started a compilation timeout timer of 180 seconds.
Compilation child process completed within timeout period.
Compilation failed! 
Try to compile segment with 110 ops

As a note the edge compilation just ends at the 110 mark. No warnings or other stuff it just ends.

I assume this is probably due to the quantized_model being roughly 32MB Which makes me assume that the TF2 pretrained models are just too big unless they are the EfficientDet models. Would this be a correct assumption?

Not able to upload the model due to getting a file to big error message, but the steps above should allow one to recreate the issue.

[hjonnala]

Thanks for the steps.. Could you please share the tflite file as well..Thanks!!

[zcase]

@hjonnala the quantized model is 32MB and the zip file is like 28MB which is to big to upload. Github doesn't seem to like the tar.xz file so I'm not able to share it. Again the steps above are just using the pretrained TF2 models as if they were the custom model. (no additional custom training was done)

[hjonnala]

can you add the tflite file to the google drive and share the link please.

[zcase]

@hjonnala I am on a system that does not have access to google drive. I think you may just need to follow my steps above to reproduce the tflite file.

[hjonnala]

Ok.. I have tried you colab and able to compile the SSD MobileNet V1 FPN 640x640 tflite model.

[google-coral-bot[bot]]

Are you satisfied with the resolution of your issue? Yes No