Regeneration of YOLOX model

[PINTO0309]

1. OS you are using e.g. Ubuntu 20.04, WIndows10, etc: Ubuntu20.04

2. OS Architecture e.g. x86_64, armv7l, aarch64, etc: x86_64

3. Version of OpenVINO e.g. 2021.2.185, etc: 2021.4.582

4. Version of TensorFlow e.g. v2.4.1, tf-nightly==2.5.0.dev20210128, etc: v2.6.0-rc1

5. Version of TensorRT e.g. TensorRT6.0 GA, etc: 8.0

9. URL of the repository from which the transformed model was taken

https://github.com/PINTO0309/PINTO_model_zoo/tree/main/132_YOLOX

11. Issue Details

Regenerate and commit the model with the modifications of https://github.com/PINTO0309/openvino2tensorflow/commit/f037817dcaeebe9208293dd598d93f39d7d13602 and https://github.com/PINTO0309/openvino2tensorflow/commit/963fa53a52bd3e7bb6f17784037d395a0f596eee.

[NobuoTsukamoto]

Integer quantization model conversion fails (nano, tiny).
This is probably due to the model, not the openvino2tensorflow issue. The error is:

  Integer Quantization started ========================================================
  WARNING:absl:Importing a function (__inference__wrapped_model_4389) with ops with unsaved custom gradients. Will likely fail if a gradient is requested.
  WARNING:absl:Importing a function (__inference_model_layer_call_and_return_conditional_losses_14552) with ops with unsaved custom gradients. Will likely fail if a gradient is requested.
  WARNING:absl:Importing a function (__inference_model_layer_call_and_return_conditional_losses_11454) with ops with unsaved custom gradients. Will likely fail if a gradient is requested.
  WARNING:absl:Importing a function (__inference_model_layer_call_and_return_conditional_losses_13233) with ops with unsaved custom gradients. Will likely fail if a gradient is requested.
  WARNING:absl:Importing a function (__inference_model_layer_call_and_return_conditional_losses_8610) with ops with unsaved custom gradients. Will likely fail if a gradient is requested.
  fully_quantize: 0, inference_type: 6, input_inference_type: 0, output_inference_type: 0
  loc("model/tf.concat_14/concat"): error: 'tfl.concatenation' op quantization parameters violate the same scale constraint: !quant.uniform<i8:f32, 0.031797535717487335:-119> vs. !quant.uniform<i8:f32, 0.034242726862430573:-120>
  ERROR: Failed to quantize: 
  Traceback (most recent call last):
    File "/usr/local/bin/openvino2tensorflow", line 2646, in convert
      tflite_model = converter.convert()
    File "/usr/local/lib/python3.8/dist-packages/tensorflow/lite/python/lite.py", line 729, in wrapper
      return self._convert_and_export_metrics(convert_func, *args, **kwargs)
    File "/usr/local/lib/python3.8/dist-packages/tensorflow/lite/python/lite.py", line 715, in _convert_and_export_metrics
      result = convert_func(self, *args, **kwargs)
    File "/usr/local/lib/python3.8/dist-packages/tensorflow/lite/python/lite.py", line 996, in convert
      return self._optimize_tflite_model(
    File "/usr/local/lib/python3.8/dist-packages/tensorflow/lite/python/convert_phase.py", line 218, in wrapper
      raise error from None  # Re-throws the exception.
    File "/usr/local/lib/python3.8/dist-packages/tensorflow/lite/python/convert_phase.py", line 208, in wrapper
      return func(*args, **kwargs)
    File "/usr/local/lib/python3.8/dist-packages/tensorflow/lite/python/lite.py", line 689, in _optimize_tflite_model
      model = self._quantize(
    File "/usr/local/lib/python3.8/dist-packages/tensorflow/lite/python/lite.py", line 520, in _quantize
      return _mlir_quantize(
    File "/usr/local/lib/python3.8/dist-packages/tensorflow/lite/python/convert_phase.py", line 218, in wrapper
      raise error from None  # Re-throws the exception.
    File "/usr/local/lib/python3.8/dist-packages/tensorflow/lite/python/convert_phase.py", line 208, in wrapper
      return func(*args, **kwargs)
    File "/usr/local/lib/python3.8/dist-packages/tensorflow/lite/python/convert.py", line 236, in mlir_quantize
      return wrap_toco.wrapped_experimental_mlir_quantize(
    File "/usr/local/lib/python3.8/dist-packages/tensorflow/lite/python/wrap_toco.py", line 47, in wrapped_experimental_mlir_quantize
      return _pywrap_toco_api.ExperimentalMlirQuantizeModel(
  RuntimeError: Failed to quantize: 
  All the conversion process is finished! =============================================

It is presumed that this is due to the different input ranges of the concatenated ope. Therefore, it seems that it cannot be converted to an Integer quantization model due to model constraints.

  loc("model/tf.concat_14/concat"): error: 'tfl.concatenation' op quantization parameters violate the same scale constraint: !quant.uniform<i8:f32, 0.031797535717487335:-119> vs. !quant.uniform<i8:f32, 0.034242726862430573:-120>

[PINTO0309]

@NobuoTsukamoto Thank you. I haven't tried it yet, but I can think of three ways.

1. Downgrade to TensorFlow v2.4.1 and try
2. Multiply normalization during quantization by 1 (Calibration parameters)
3. uint8 -> int8

However, conviction is not the way to be popular.

[NobuoTsukamoto]

It seems that the error can be avoided by setting experimental_new_quantizer = False.

Later, I would like to share the code.

[PINTO0309]

Fixes. v1.16.1

• Added --use_experimental_new_quantizer option (Default: False)
• Change EdgeTPU compiler option from -sa to -sad https://github.com/PINTO0309/openvino2tensorflow/releases/tag/v1.16.1

[NobuoTsukamoto]

@PINTO0309 -san Thanks!!

[NobuoTsukamoto]

Post-quantization training with experimental_new_quantizer = False reduced accuracy. There may be model constraints.

YOLOX-Nano 416x416

 Average Precision  (AP) @[ IoU=0.50:0.95 | area=   all | maxDets=100 ] = 0.064
 Average Precision  (AP) @[ IoU=0.50      | area=   all | maxDets=100 ] = 0.118
 Average Precision  (AP) @[ IoU=0.75      | area=   all | maxDets=100 ] = 0.063
 Average Precision  (AP) @[ IoU=0.50:0.95 | area= small | maxDets=100 ] = 0.018
 Average Precision  (AP) @[ IoU=0.50:0.95 | area=medium | maxDets=100 ] = 0.064
 Average Precision  (AP) @[ IoU=0.50:0.95 | area= large | maxDets=100 ] = 0.120
 Average Recall     (AR) @[ IoU=0.50:0.95 | area=   all | maxDets=  1 ] = 0.099
 Average Recall     (AR) @[ IoU=0.50:0.95 | area=   all | maxDets= 10 ] = 0.154
 Average Recall     (AR) @[ IoU=0.50:0.95 | area=   all | maxDets=100 ] = 0.166
 Average Recall     (AR) @[ IoU=0.50:0.95 | area= small | maxDets=100 ] = 0.031
 Average Recall     (AR) @[ IoU=0.50:0.95 | area=medium | maxDets=100 ] = 0.156
 Average Recall     (AR) @[ IoU=0.50:0.95 | area= large | maxDets=100 ] = 0.306

[PINTO0309]

Fixes: 74f3ca68cc785a143380a44f4028d47dfa54c928