Open MATTYGILO opened 2 years ago
Same problem with mobilenetlarge:
`AttributeError: Exception encountered when calling layer "tf.operators.add_29" (type TFOpLambda).
'list' object has no attribute 'dtype'
Call arguments received by layer "tf.operators.add_29" (type TFOpLambda): • x=['tf.Tensor(shape=(None, None, None, 16), dtype=float32)'] • y=3.0 • name=None`
Ok now I am setting minimalist to True and getting this error:
RuntimeError: Layer rescaling_2:<class 'keras.layers.preprocessing.image_preprocessing.Rescaling'> is not supported. You can quantize this layer by passing a
tfmot.quantization.keras.QuantizeConfiginstance to the
quantize_annotate_layerAPI.
Ok this seems to work:
model = MobileNetV3Large(
input_shape=(224, 224, 3),
include_top=True,
weights=None,
minimalistic=True, <-- Include this
include_preprocessing=False, <-- Include this
)
Additionally, I have noticed that this also seems to work for MobileNetV3Small even though in the article earlier it only states MobileNetV3Large
Ok now I run this code:
import numpy as np
import tensorflow as tf
def representative_dataset():
for _ in range(1000):
data = np.random.choice([0, 255], size=(1, 224, 224, 1))
yield [data.astype(np.float32)]
converter = tf.lite.TFLiteConverter.from_keras_model(qat_model)
converter.experimental_new_converter = True
converter.optimizations = [tf.lite.Optimize.DEFAULT]
converter.representative_dataset = representative_dataset
converter.target_spec.supported_ops = [tf.lite.OpsSet.TFLITE_BUILTINS_INT8]
converter.inference_input_type = tf.int8 # or tf.uint8
converter.inference_output_type = tf.int8 # or tf.uint8
tflite_quant_model_bytes = converter.convert()
open("testQuant.tflite", "wb").write(tflite_quant_model_bytes)
and get this error:
RuntimeError: tensorflow/lite/kernels/conv.cc:357 input_channel % filter_input_channel != 0 (1 != 0)Node number 1 (CONV_2D) failed to prepare.
for both int8 and uint8.
I have ran it without full int 8 quantisation and it works. But I need full int 8!!!
Hi the model input shapes are (224, 224, 3), so you can try size=(1, 224, 224, 3) on representative_dataset function. Thanks.
There has recently been an article saying mobilenetv3 can be used for QAT https://blog.tensorflow.org/2022/06/Adding-Quantization-aware-Training-and-Pruning-to-the-TensorFlow-Model-Garden.html
However when I run this code:
I get this error:
I am using an m1 Mac: conda Python 3.10.4 tensorflow-macos 2.9.2 tensorflow-metal 0.5.0 tensorflow-model-optimization 0.7.2