Is there an easy way to convert ONNX or PB from (NCHW) to (NHWC)?

[AlexeyAB]

@PINTO0309 Hi, Nice work with YOLOv4 / tiny!

As I see you use:

• NCHW for: OpenVINO (xml / bin), Darknet (cfg / weights)

• NHWC for: TFLite, Keras (yolov4_tiny_voc.json / yolov4_tiny_voc.h5), TF1 (pb), TF2 (saved_models.json / saved_models.pb)

I have several questions:

• Is there an easy way to convert ONNX or PB from (NCHW) to (NHWC)? I've seen converters that add transpose before and after each layer, but this seems to slow things down a lot. Is it possible to do this transformation without slowing down the inference?

• Is there an easy way to convert TF1-pb to TF2-saved_models.pb ?

• Is NHWC slowing down execution on the GPU?

• How many FPS do you get on Google Coral TPU-Edge and RaspberryPi4 for yolov4-tiny (int8)?

• What script did you use to get yolov4_tiny_voc.json ?

[PINTO0309]

@itsmasabdi

Since output_weight_quant_tflite only quantizes the weights, it cannot be converted to the edgetpu model. edgetpu needs to quantize all the OPs and perform Full Integer Quantization, so it must be converted according to the following procedure.

$ openvino2tensorflow \
  --model_path={model_path} \
  --output_saved_model True

• quantization.py

  
  import tensorflow as tf
  import tensorflow_datasets as tfds
  import numpy as np

def representative_dataset_gen(): for data in raw_test_data.take(10): image = data['image'].numpy() image = tf.image.resize(image, (416, 416)) image = image[np.newaxis,:,:,:] image = image / 255. yield [image]

raw_test_data, info = tfds.load(name="voc/2007", with_info=True, split="validation", data_dir="~/TFDS", download=True)

Full Integer Quantization - Input/Output=float32

converter = tf.lite.TFLiteConverter.from_saved_model('saved_model') converter.optimizations = [tf.lite.Optimize.DEFAULT] converter.target_spec.supported_ops = [tf.lite.OpsSet.TFLITE_BUILTINS_INT8,tf.lite.OpsSet.SELECT_TF_OPS] converter.representative_dataset = representative_dataset_gen converter.inference_input_type = tf.uint8 converter.inference_output_type = tf.uint8 tflite_quant_model = converter.convert() with open('yolov4_416_full_integer_quant.tflite', 'wb') as w: w.write(tflite_quant_model) print("Full Integer Quantization complete! - yolov4_416_full_integer_quant.tflite")

```bash
$ python3 quantization.py

$ edgetpu_compiler -s yolov4_416_full_integer_quant.tflite

The normalization process of representative_dataset_gen should be adjusted by you.

[itsmasabdi]

Hi @PINTO0309

Thanks for your response. I am having issues converting the model, wondering if you can help?

This time I:

• Downloaded yolov4_tiny_voc_416x416_FP32 from your repo
• Used openvino/ubuntu18_dev docker image and installed pip3 install openvino2tensorflow --upgrade
• openvino2tensorflow --model_path=yolov4_tiny_voc_416x416_FP32.xml --output_saved_model True
• Convert the resulting saved_model using quantization.py

The error I get is:

Traceback (most recent call last):
  File "quant.py", line 21, in <module>
    converter = tf.lite.TFLiteConverter.from_saved_model('saved_model')
  File "/usr/local/lib/python3.6/dist-packages/tensorflow/lite/python/lite.py", line 1069, in from_saved_model
    saved_model = _load(saved_model_dir, tags)
  File "/usr/local/lib/python3.6/dist-packages/tensorflow/python/saved_model/load.py", line 859, in load
    return load_internal(export_dir, tags, options)["root"]
  File "/usr/local/lib/python3.6/dist-packages/tensorflow/python/saved_model/load.py", line 909, in load_internal
    root = load_v1_in_v2.load(export_dir, tags)
  File "/usr/local/lib/python3.6/dist-packages/tensorflow/python/saved_model/load_v1_in_v2.py", line 279, in load
    return loader.load(tags=tags)
  File "/usr/local/lib/python3.6/dist-packages/tensorflow/python/saved_model/load_v1_in_v2.py", line 204, in load
    meta_graph_def = self.get_meta_graph_def_from_tags(tags)
  File "/usr/local/lib/python3.6/dist-packages/tensorflow/python/saved_model/load_v1_in_v2.py", line 87, in get_meta_graph_def_from_tags
    tags)
  File "/usr/local/lib/python3.6/dist-packages/tensorflow/python/saved_model/loader_impl.py", line 358, in get_meta_graph_def_from_tags
    "\navailable_tags: " + str(available_tags))
RuntimeError: MetaGraphDef associated with tags {'serve'} could not be found in SavedModel. To inspect available tag-sets in the SavedModel, please use the SavedModel CLI: `saved_model_cli`
available_tags: []

I tried different tensorflow version but can't get passed this error

I would appreciate your help

[PINTO0309]

@itsmasabdi You should use the Docker environment. https://github.com/PINTO0309/openvino2tensorflow#4-setup

[Fafa87]

Just info for someone that does not like openvino path:

• there is also a path through keras (onnx2keras with change ordering parameter)
• you may need to tweak onnx2keras to get to to work but I managed to get tfjs model (without transpose mess) that takes 20ms instead of 68ms which is very nice :)