search

AxisCommunications / acap-native-sdk-examples

Example code for APIs and features in AXIS Camera Application Platform (ACAP) Native SDK
Apache License 2.0
41 stars 23 forks source link

WARNING: Fallback unsupported op 32 to TfLite #185

Closed KushalHM closed 7 months ago

KushalHM commented 7 months ago

Running SSD mobilenetv2 I get in the log the following warning: Fallback unsupported op 32 to TFlite

Axis Device Model: AXIS P3267-LVE Firmware: 11.6.94 Model Architecture: Mobilenet V2 SSD (object detection) Model Resolution: Custom, 16:9 aspect ratio TFlite Details: Quantize Aware Trained, TF1, with per tensor Quantization.

Reproduction steps:

  1. Load the ACAP with the above tflite model.
  2. Run the app and as soon as model is loaded, the above warning message appears.
pataxis commented 7 months ago

Hi @KushalHM , thanks for your report. Is the ACAP application the object-detection example built for Artpec8 without any changes or is it a custom application?

KushalHM commented 7 months ago

Hi @KushalHM , thanks for your report. Is the ACAP application the object-detection example built for Artpec8 without any changes or is it a custom application?

Hi @pataxis , Thanks for the quick reply. This is a custom application based on the object detection example, but the issue occurs when larod loads the model. The issue occurs with the custom mobilenet ssd v2 model as well as the default one (300x300) available in the TF1 or Coral model zoo.

So I am assuming even the object-detection example with this model (https://raw.githubusercontent.com/google-coral/test_data/master/ssd_mobilenet_v2_coco_quant_postprocess.tflite) would result in the warning.

Corallo commented 7 months ago

Hi @KushalHM, That log is only warning you that your network contains a layer that cannot be processed by the DLPU. It is talking about the last post-processing layer that outputs float32. Because that is not supported by the DLPU, it will fallback to the cpu.

It is not a bug, but an intended feature.

KushalHM commented 7 months ago

Hi @KushalHM, That log is only warning you that your network contains a layer that cannot be processed by the DLPU. It is talking about the last post-processing layer that outputs float32. Because that is not supported by the DLPU, it will fallback to the cpu.

It is not a bug, but an intended feature.

Hi @Corallo ,

Understood, thanks for the clarification. Just a final question regarding the same, does this fallback operation delay the loading of model for the first time?

We have noticed about a minute of delay in loading the model when model is not cached, when application started for the first time. For subsequent restarts, the model is loaded within a couple of seconds.

Corallo commented 7 months ago

You will get a slow first inference in any case the case, it doesn't matter if there is a layer that falls back to the CPU or not. That is because the tflite model is compiled during the first inference. Then the compiled model is cached, and as you say, you will not see that delay next time you load the model.