No tf_py_to_tflite converter for tf.nn.batch_normalization

[idenc]

When I try to convert any model with batch normalizations (e.g. mobilenet) from NNEF To TFLite I get AssertionError: No tf_py_to_tflite converter for tf.nn.batch_normalization. Should the batch normalizations be folded out?

[leehangyu]

same problem here,

pytorch to onnx to nnef to tflite failed in tf_py_to_tflite file because of batch_normalization.

Edit:

There is no sigmoid operation either

[gyenesvi]

Yes, TFLite does not support the batch normalization op, and it seems there is no sigmoid either. By folding out, do you mean folding it into the previous convolution weights?

[idenc]

Yes, that's what the TFLite converter does I believe

[jnorwood]

There are batch normalization ops in the NNEF model translated from the fp32 inception v3, in the NNEF models on github.

I believe this operation is only intended to be used during training. Should the operation be optimized out of the NNEF model translation?

I see the tflite quantized translation does exclude batch normalization, probably folding this into the weights as the other posters have suggested.

I haven't looked at the fp32 tf implementation to see if they do the same. There is some discussion here: https://www.intel.com/content/www/us/en/artificial-intelligence/posts/improving-tensorflow-inference-performance-on-intel-xeon-processors.html

[gyenesvi]

The NNEF converter has been updated, and when used with the --optimize flag, the batch norm ops should disappear. If you already have an NNEF model with batch norms in it, you can also use the converter to convert from NNEF to NNEF with optimization turned on, and you should get an NNEF model without batch norms. Afterwards, you should be able to convert it to TFLite. Let me know if I can close the issue.

[jnorwood]

Should the model zoo be updated by applying the --optimize to remove the batch_normalization calls as part of this issue? For example, Inception V4 still contains all the batch norms.

[gyenesvi]

I'd prefer keeping the unoptimized models in the zoo, just to have a clearer 1-1 mapping between the original and converted models as a baseline. If you need an optimized model, you can always run the optimizer on the converted NNEF model itself:

python -m nnef_tools.convert --input-format nnef --output-format nnef --input-model --optimize path/to/model.nnef