142_HITNET model conversion to edgetpu

[Ric92]

Issue Type

Support

OS

Ubuntu

OS architecture

x86_64

Programming Language

Python

Framework

TensorFlowLite

Description

I have a problem converting the HITNET model to edgeTPU in particular when converting the saved_model to full_integer_quant. I have already seen that it is not by default converted and I don't know if it is something that definitely can't be done. I tested eth3d, flyingthings finalpass xl, and middlebury d400.

Thanks in advance

Relevant Log Output

Start conversion process from saved_model to tflite ======================================
Before changing the input shape Tensor("input:0", shape=(None, None, None, 2), dtype=float32)
After  changing the input shape Tensor("input:0", shape=(1, 120, 160, 2), dtype=float32)
numpy dataset load started ==========================================================
numpy dataset load complete!
Full Integer Quantization started ===================================================
WARNING:absl:Please consider providing the trackable_obj argument in the from_concrete_functions. Providing without the trackable_obj argument is deprecated and it will use the deprecated conversion path.
Estimated count of arithmetic ops: 3.500 G  ops, equivalently 1.750 G  MACs
INFO: Created TensorFlow Lite delegate for select TF ops.
INFO: TfLiteFlexDelegate delegate: 20 nodes delegated out of 774 nodes with 10 partitions.

ERROR: operands could not be broadcast together with shapes (1,120,160,2) (3,) 
Traceback (most recent call last):
  File "/usr/local/bin/saved_model_to_tflite", line 278, in convert
    tflite_model = converter.convert()
  File "/usr/local/lib/python3.8/dist-packages/tensorflow/lite/python/lite.py", line 1714, in convert
    return super(TFLiteConverterV2, self).convert()
  File "/usr/local/lib/python3.8/dist-packages/tensorflow/lite/python/lite.py", line 803, in wrapper
    return self._convert_and_export_metrics(convert_func, *args, **kwargs)
  File "/usr/local/lib/python3.8/dist-packages/tensorflow/lite/python/lite.py", line 789, in _convert_and_export_metrics
    result = convert_func(self, *args, **kwargs)
  File "/usr/local/lib/python3.8/dist-packages/tensorflow/lite/python/lite.py", line 1383, in convert
    return super(TFLiteFrozenGraphConverterV2,
  File "/usr/local/lib/python3.8/dist-packages/tensorflow/lite/python/lite.py", line 1009, in convert
    return self._optimize_tflite_model(
  File "/usr/local/lib/python3.8/dist-packages/tensorflow/lite/python/convert_phase.py", line 216, in wrapper
    raise error from None  # Re-throws the exception.
  File "/usr/local/lib/python3.8/dist-packages/tensorflow/lite/python/convert_phase.py", line 206, in wrapper
    return func(*args, **kwargs)
  File "/usr/local/lib/python3.8/dist-packages/tensorflow/lite/python/lite.py", line 749, in _optimize_tflite_model
    model = self._quantize(
  File "/usr/local/lib/python3.8/dist-packages/tensorflow/lite/python/lite.py", line 555, in _quantize
    calibrated = calibrate_quantize.calibrate(
  File "/usr/local/lib/python3.8/dist-packages/tensorflow/lite/python/convert_phase.py", line 216, in wrapper
    raise error from None  # Re-throws the exception.
  File "/usr/local/lib/python3.8/dist-packages/tensorflow/lite/python/convert_phase.py", line 206, in wrapper
    return func(*args, **kwargs)
  File "/usr/local/lib/python3.8/dist-packages/tensorflow/lite/python/optimize/calibrator.py", line 224, in calibrate
    self._feed_tensors(dataset_gen, resize_input=True)
  File "/usr/local/lib/python3.8/dist-packages/tensorflow/lite/python/optimize/calibrator.py", line 93, in _feed_tensors
    for sample in dataset_gen():
  File "/usr/local/bin/saved_model_to_tflite", line 238, in representative_dataset_gen
    tmp_image = eval(string_formulas_for_normalization) # Default: (data - [127.5,127.5,127.5]) / [127.5,127.5,127.5]
  File "<string>", line 1, in <module>
ValueError: operands could not be broadcast together with shapes (1,120,160,2) (3,) 
EdgeTPU convertion started ==========================================================
ERROR: 
Traceback (most recent call last):
  File "/usr/local/bin/saved_model_to_tflite", line 291, in convert
    result = subprocess.check_output(
  File "/usr/lib/python3.8/subprocess.py", line 415, in check_output
    return run(*popenargs, stdout=PIPE, timeout=timeout, check=True,
  File "/usr/lib/python3.8/subprocess.py", line 516, in run
    raise CalledProcessError(retcode, process.args,
subprocess.CalledProcessError: Command '['edgetpu_compiler', '-o', 'eth3d/saved_model_120x160', '-sad', '-t', '3600', '-n', '1', 'eth3d/saved_model_120x160/model_full_integer_quant.tflite']' returned non-zero exit status 1.
--------------------------------------------------------------------------------
Please install edgetpu_compiler according to the following website.
https://coral.ai/docs/edgetpu/compiler/#system-requirements
All the conversion process is finished! =============================================

URL or source code for simple inference testing code

No response

[PINTO0309]

First, the conversion command you ran is wrong. You are trying to do a 3-channel normalization, and it looks like the image in the calibration dataset has 3 channels. The input is expected to be two channels.

tmp_image = eval(string_formulas_for_normalization) # Default: (data - [127.5,127.5,127.5]) / [127.5,127.5,127.5]
↓
ValueError: operands could not be broadcast together with shapes (1,120,160,2) (3,)

Secondly, even if the calibration is done correctly, the conversion to EdgeTPU is not possible at this time. Before converting to TPU, there are two types of operations that are not supported by TensorFlow Lite. These are StridedSlice in 6D and Transpose in 7D.

I have asked the TensorFlow team to see if they can solve this problem for me. However, I think they are very busy and it will be many years before they will heed my wishes. https://github.com/tensorflow/tensorflow/issues/53702

[Ric92]

Thank you very much for your answer, we will wait for the TensorFlow team then.

Congratulations for your work.