Optimize TF Lite model?

[don]

@sandeepmistry Did you try quantization on the TFLite model?

[sandeepmistry]

I'll try it again today. Before it was slower and not reliable, then at one point hung the board.

Maybe things will be better with normalized inputs now :)

[sandeepmistry]

It's crashing today again, using this in the notebook:

converter = tf.lite.TFLiteConverter.from_keras_model(model)
converter.optimizations = [tf.lite.Optimize.OPTIMIZE_FOR_SIZE]
tflite_model = converter.convert()
# Save the model to disk
open("gesture_model_quantized.tflite", "wb").write(tflite_model)

[sandeepmistry]

tf.lite.Optimize.DEFAULT also crashes, I'll try again with the latest TFLM nightly zip

[sandeepmistry]

The assert is failing here:

// Correctly-rounded-to-nearest division by a power-of-two.
// Also known as a rounding arithmetic right shift.
template <typename IntegerType>
inline IntegerType RoundingDivideByPOT(IntegerType x, int exponent) {
  assert(exponent >= 0);
  assert(exponent <= 31);
  const IntegerType mask = Dup<IntegerType>((1ll << exponent) - 1);
  const IntegerType zero = Dup<IntegerType>(0);
  const IntegerType one = Dup<IntegerType>(1);
  const IntegerType remainder = BitAnd(x, mask);
  const IntegerType threshold =
      Add(ShiftRight(mask, 1), BitAnd(MaskIfLessThan(x, zero), one));
  return Add(ShiftRight(x, exponent),
             BitAnd(MaskIfGreaterThan(remainder, threshold), one));
}

src/third_party/gemmlowp/fixedpoint/fixedpoint.h:359: assert(exponent <= 31);

[sandeepmistry]

Still crashing today, with the model that has softmax on the output layer :(

[don]

wontfix