Trying to quantise MobileNetv3 small Exception encountered when calling layer "tf.__operators__.add_137"

[MATTYGILO]

There has recently been an article saying mobilenetv3 can be used for QAT https://blog.tensorflow.org/2022/06/Adding-Quantization-aware-Training-and-Pruning-to-the-TensorFlow-Model-Garden.html

However when I run this code:

from tensorflow.keras.applications import MobileNetV3Small
import tensorflow_model_optimization as tfmot

model = MobileNetV3Small(
    input_shape=(224, 224, 3),
    include_top=True,
    weights=None,
    classes=200,
)

qat_model = tfmot.quantization.keras.quantize_model(model)

I get this error:

AttributeError: Exception encountered when calling layer "tf.__operators__.add_56" (type TFOpLambda).

'list' object has no attribute 'dtype'

Call arguments received by layer "tf.__operators__.add_56" (type TFOpLambda):
  • x=['tf.Tensor(shape=(None, 112, 112, 16), dtype=float32)']
  • y=3.0
  • name=None

I am using an m1 Mac: conda Python 3.10.4 tensorflow-macos 2.9.2 tensorflow-metal 0.5.0 tensorflow-model-optimization 0.7.2

[MATTYGILO]

Same problem with mobilenetlarge:

`AttributeError: Exception encountered when calling layer "tf.operators.add_29" (type TFOpLambda).

'list' object has no attribute 'dtype'

Call arguments received by layer "tf.operators.add_29" (type TFOpLambda): • x=['tf.Tensor(shape=(None, None, None, 16), dtype=float32)'] • y=3.0 • name=None`

[MATTYGILO]

Ok now I am setting minimalist to True and getting this error:

RuntimeError: Layer rescaling_2:<class 'keras.layers.preprocessing.image_preprocessing.Rescaling'> is not supported. You can quantize this layer by passing atfmot.quantization.keras.QuantizeConfiginstance to thequantize_annotate_layerAPI.

[MATTYGILO]

Ok this seems to work:

model = MobileNetV3Large(
    input_shape=(224, 224, 3),
    include_top=True,
    weights=None,
    minimalistic=True, <-- Include this
    include_preprocessing=False, <-- Include this
)

[MATTYGILO]

Additionally, I have noticed that this also seems to work for MobileNetV3Small even though in the article earlier it only states MobileNetV3Large

[MATTYGILO]

Ok now I run this code:

import numpy as np
import tensorflow as tf

def representative_dataset():
    for _ in range(1000):
        data = np.random.choice([0, 255], size=(1, 224, 224, 1))
        yield [data.astype(np.float32)]

converter = tf.lite.TFLiteConverter.from_keras_model(qat_model)

converter.experimental_new_converter = True
converter.optimizations = [tf.lite.Optimize.DEFAULT]
converter.representative_dataset = representative_dataset
converter.target_spec.supported_ops = [tf.lite.OpsSet.TFLITE_BUILTINS_INT8]
converter.inference_input_type = tf.int8  # or tf.uint8
converter.inference_output_type = tf.int8  # or tf.uint8

tflite_quant_model_bytes = converter.convert()
open("testQuant.tflite", "wb").write(tflite_quant_model_bytes)

and get this error:

RuntimeError: tensorflow/lite/kernels/conv.cc:357 input_channel % filter_input_channel != 0 (1 != 0)Node number 1 (CONV_2D) failed to prepare.

for both int8 and uint8.

I have ran it without full int 8 quantisation and it works. But I need full int 8!!!

[Xhark]

Hi the model input shapes are (224, 224, 3), so you can try size=(1, 224, 224, 3) on representative_dataset function. Thanks.