TFLite Hexagon Delegate accuracy with Depthwise conv 7x7 filter

[tensorbuffer]

Describe the problem

We have a model that has accuracy issue with depthwiseconv, I attached two models, one is right before the depthwise conv node, the result of DSP and CPU has small difference (~0.2), another model has a depthwise conv (7x7 filter) and the result difference is huge (~18.). I see that there was an issue before with 5x5 filter that got a workaround last year. I am guessing it has the same issue with 7x7 filter: https://github.com/tensorflow/tensorflow/issues/54481 We have qualcomm NDA and we have the source code of nnlib (they don't release update now), wondering if you know where the bug is in nnlib? If so it's best to modify in nnlib side, otherwise maybe you can modify your workaround toward the 7x7 filter?

Source code / logs

dwconv_debug.zip

[sushreebarsa]

@tensorbuffer Could you use Post-training quantization techniques instead of quantization-aware training. Also using Integer Quantization (INT8) instead of Float16 (FP16) quantization could help. Please let us know the outcome. Thank you!

[tensorbuffer]

Yes that's how this model is generated (post training quantization, and int8): converter = tf.lite.TFLiteConverter.from_keras_model(model) converter.optimizations = [tf.lite.Optimize.DEFAULT] converter.representative_dataset = representative_dataset_gen_rand converter.target_spec.supported_ops = [tf.lite.OpsSet.TFLITE_BUILTINS_INT8] tflite_quant_model = converter.convert()

You can just take the model and run on CPU and DSP to compare the result, feed it with a random input. You can see the model before depthwiseconv node has similar outputs, and the model with the depthwiseconv node has quite different results.

[LakshmiKalaKadali]

Hi @pkgoogle,

Please look into the issue.

Thank You

[pkgoogle]

Hi @sirakiin, can you please take a look? Thanks.