the difference between tflite quantize output and tflite micro quantize output

[magnethead117]

I am comparing the difference of tflite output and tflite micro(esp32: https://github.com/espressif/esp-tflite-micro) output layer by layer. The first layer is quantize just as the image shows: The first 64 number of this layer is : [ 50 48 47 48 50 56 64 76 90 93 91 89 86 83 81 81 81 80 80 81 82 85 84 86 86 87 87 89 88 89 89 89 90 91 92 92 93 92 93 94 94 96 97 99 101 105 108 115 121 123 125 129 131 134 136 137 138 139 140 140 139 133 129 122] And corresponding output of this laryer (tflite) is, which is pretty strange: [ 15 -15 -128 -95 -113 -102 -106 -89 -128 -126 -128 -128 -116 -101 -121 -128 -117 -86 -99 -94 -128 -106 -110 -120 -102 -96 -121 -110 -119 -100 -128 -127 -99 -116 -118 -102 -128 -128 -126 -104 -111 -128 -90 -103 -118 -124 -128 -108 -128 -109 -128 -128 -122 -112 -125 -120 -128 -128 -89 -128 -128 -119 -122 -127] before_quant = inter.get_tensor(0) print(before_quant.reshape(-1,)[:64]) after_quant = inter.get_tensor(15) print(after_quant.reshape(-1,)[:64]) Using tflite micro, I got a totally different res: [-78, -80, -81, -80, -78, -72, -64, -52, -38, -35, -37, -39, -42, -45, -47, -47, -47, -48, -48, -47, -46, -43, -44, -42, -42, -41, -41, -39 -43, -44, -42, -42, -41, -41, -39, -40, -39, -39, -39, -38, -37, -36, -36, -35, -36, -35, -34, -34, -329, -27, - -2 3, 6, 8, 9, 10, 11, 1, -40, -39, -39, -39, -38, -37, -36, -36, -35, -36, -35, -34, -34, -32, -31, -29, -27, -23, -20, -13, -7, -5, -3, 1, 3, 6, 8, 9,, -31, -29, -27, -23, -20, -13, -7, -5, -3, 1, 3, 6, 8, 9, 10, 11, 12, 12, 11, 5, 1, -6] it seems like input-128

Two questions:

1. why there is the difference between tflite quantize output and tflite micro quantize output ?
2. the micro lite output seems more resonable, how to get the tflite output? I didn' find the corresponding codes in python's tensorflow package.

I would be grateful for your response at your earliest convenience.

[ddavis-2015]

@magnethead117

In your code, are you doing something similar to the following:

  tflm_interpreter = runtime.Interpreter.from_file(
      _INPUT_TFLITE_FILE.value,
      intrepreter_config=runtime.InterpreterConfig.kPreserveAllTensors,
  )

  tflite_interpreter = tf.lite.Interpreter(
      model_path=_INPUT_TFLITE_FILE.value,
      experimental_preserve_all_tensors=True,
  )

Without the tensor preservation flags, you may get unexpected results when inspecting the output of a layer.

[magnethead117]

Thank you, problems solved

@magnethead117

In your code, are you doing something similar to the following:

  tflm_interpreter = runtime.Interpreter.from_file(
      _INPUT_TFLITE_FILE.value,
      intrepreter_config=runtime.InterpreterConfig.kPreserveAllTensors,
  )

  tflite_interpreter = tf.lite.Interpreter(
      model_path=_INPUT_TFLITE_FILE.value,
      experimental_preserve_all_tensors=True,
  )

Without the tensor preservation flags, you may get unexpected results when inspecting the output of a layer.