Google Coral EdgeTPU

[jurenovic]

Hey @YunYang1994, do you think its possible to convert Yolo3 model into tflite model that can be run on a coral edge tpu.

From docs:

You need to convert your model to TensorFlow Lite and it must be quantized using either quantization-aware training. https://coral.ai/docs/edgetpu/

Thanks for your answer...

[YunYang1994]

absolutely yes !

[jurenovic]

@YunYang1994 would you be so kind to point me into the right direction. I created a simple python script to do the conversion, but its failing with a Fatal Python error.

import tensorflow as tf
from core.yolov3 import YOLOv3, decode

def representative_dataset_gen():
  for _ in range(num_calibration_steps):
    # Get sample input data as a numpy array in a method of your choosing.
    yield [input]

input_size   = 416

input_layer  = tf.keras.layers.Input([input_size, input_size, 3])
feature_maps = YOLOv3(input_layer)

bbox_tensors = []
for i, fm in enumerate(feature_maps):
    bbox_tensor = decode(fm, i)
    bbox_tensors.append(bbox_tensor)

model = tf.keras.Model(input_layer, bbox_tensors)
utils.load_weights(model, "./yolov3.weights")
model.summary()

# Create a converter
converter = tf.lite.TFLiteConverter.from_keras_model(model)
# Convert the model
# converter.optimizations = [tf.lite.Optimize.DEFAULT]
# converter.representative_dataset = representative_dataset_gen
# converter.target_spec.supported_ops = [tf.lite.OpsSet.TFLITE_BUILTINS_INT8]
# converter.inference_input_type = tf.uint8
# converter.inference_output_type = tf.uint8

tflite_model = converter.convert()
# Create the tflite model file
tflite_model_name = "yolov3.tflite"
open(tflite_model_name, "wb").write(tflite_model)

The exception is:

2019-12-04 22:44:51.903243: I tensorflow/core/grappler/devices.cc:60] Number of eligible GPUs (core count >= 8, compute capability >= 0.0): 0 (Note: TensorFlow was not compiled with CUDA support)
2019-12-04 22:44:51.903322: I tensorflow/core/grappler/clusters/single_machine.cc:356] Starting new session
2019-12-04 22:44:51.956138: I tensorflow/core/grappler/optimizers/meta_optimizer.cc:716] Optimization results for grappler item: graph_to_optimize
2019-12-04 22:44:51.956163: I tensorflow/core/grappler/optimizers/meta_optimizer.cc:718]   function_optimizer: function_optimizer did nothing. time = 0.007ms.
2019-12-04 22:44:51.956170: I tensorflow/core/grappler/optimizers/meta_optimizer.cc:718]   function_optimizer: function_optimizer did nothing. time = 0ms.
2019-12-04 22:45:00.864649: I tensorflow/core/grappler/devices.cc:60] Number of eligible GPUs (core count >= 8, compute capability >= 0.0): 0 (Note: TensorFlow was not compiled with CUDA support)
2019-12-04 22:45:00.864756: I tensorflow/core/grappler/clusters/single_machine.cc:356] Starting new session
2019-12-04 22:45:06.669028: I tensorflow/core/grappler/optimizers/meta_optimizer.cc:716] Optimization results for grappler item: graph_to_optimize
2019-12-04 22:45:06.669057: I tensorflow/core/grappler/optimizers/meta_optimizer.cc:718]   constant folding: Graph size after: 884 nodes (-370), 2039 edges (-370), time = 3086.21191ms.
2019-12-04 22:45:06.669063: I tensorflow/core/grappler/optimizers/meta_optimizer.cc:718]   constant folding: Graph size after: 884 nodes (0), 2039 edges (0), time = 1459.73096ms.
Traceback (most recent call last):
  File "convert.py", line 34, in <module>
    tflite_model = converter.convert()
  File "/workspace/sandbox/TensorFlow2.0-Examples/4-Object_Detection/YOLOV3/venv3/lib/python3.7/site-packages/tensorflow_core/lite/python/lite.py", line 446, in convert
    **converter_kwargs)
  File "/workspace/sandbox/TensorFlow2.0-Examples/4-Object_Detection/YOLOV3/venv3/lib/python3.7/site-packages/tensorflow_core/lite/python/convert.py", line 449, in toco_convert_impl
    enable_mlir_converter=enable_mlir_converter)
  File "/workspace/sandbox/TensorFlow2.0-Examples/4-Object_Detection/YOLOV3/venv3/lib/python3.7/site-packages/tensorflow_core/lite/python/convert.py", line 200, in toco_convert_protos
    raise ConverterError("See console for info.\n%s\n%s\n" % (stdout, stderr))
tensorflow.lite.python.convert.ConverterError: See console for info.
2019-12-04 22:45:13.865417: I tensorflow/lite/toco/graph_transformations/graph_transformations.cc:39] Before Removing unused ops: 652 operators, 1247 arrays (0 quantized)
2019-12-04 22:45:13.878738: I tensorflow/lite/toco/graph_transformations/graph_transformations.cc:39] Before general graph transformations: 652 operators, 1247 arrays (0 quantized)
2019-12-04 22:45:14.705331: F tensorflow/lite/toco/graph_transformations/resolve_strided_slice_attributes.cc:95] Check failed: start_indices_size <= num_input_axes (2 vs. 1)StridedSlice op requires no more than 1 start indices
Fatal Python error: Aborted

Current thread 0x000000010a9855c0 (most recent call first):
  File "/workspace/sandbox/TensorFlow2.0-Examples/4-Object_Detection/YOLOV3/venv3/lib/python3.7/site-packages/tensorflow_core/lite/toco/python/toco_from_protos.py", line 57 in execute
  File "/workspace/sandbox/TensorFlow2.0-Examples/4-Object_Detection/YOLOV3/venv3/lib/python3.7/site-packages/absl/app.py", line 250 in _run_main
  File "/workspace/sandbox/TensorFlow2.0-Examples/4-Object_Detection/YOLOV3/venv3/lib/python3.7/site-packages/absl/app.py", line 299 in run
  File "/workspace/sandbox/TensorFlow2.0-Examples/4-Object_Detection/YOLOV3/venv3/lib/python3.7/site-packages/tensorflow_core/python/platform/app.py", line 40 in run
  File "/workspace/sandbox/TensorFlow2.0-Examples/4-Object_Detection/YOLOV3/venv3/lib/python3.7/site-packages/tensorflow_core/lite/toco/python/toco_from_protos.py", line 94 in main
  File "/workspace/sandbox/TensorFlow2.0-Examples/4-Object_Detection/YOLOV3/venv3/bin/toco_from_protos", line 10 in <module>

I played around and with different options for the converter, but without much success.

# converter.optimizations = [tf.lite.Optimize.DEFAULT]
# converter.representative_dataset = representative_dataset_gen
# converter.target_spec.supported_ops = [tf.lite.OpsSet.TFLITE_BUILTINS_INT8]
# converter.inference_input_type = tf.uint8
# converter.inference_output_type = tf.uint8

And bear in mind that ML is completely new to me. 😄

Would appreciate some help, thanks

[jurenovic]

@YunYang1994 hey, sorry to bother you again, but would really appreciate some help

[jurenovic]

hello @YunYang1994, would really appreciate some help

[jurenovic]

@YunYang1994 any help?

[Paranormaly]

up @YunYang1994

[Namburger]

Try this: https://github.com/guichristmann/edge-tpu-tiny-yolo 👍