Tensorflow-lite segmentation model does not work with NNAPI on Android 9.0

[anilsathyan7]

System information

• OS Platform and Distribution : Android, Tensorflow-Lite-Experimental
• Mobile device : Samsung SM-A730F, Android 9 , API 28
• Tensorflow: 1.13

Describe the current behavior

The official tensorflow lite segmentation model 'deeplabv3_257_mv_gpu.tflite' cannot be run on Andorid 9.0, using NNAPI. It seems some operators are not supported currently by NNAPI.

Describe the expected behavior

The model should run on phone using NNAPI, without any problems.

Other info / logs The model gives error when we run it using tensorflow lite interpreter on android.Also it gives similar error on android benchmark tool, when we run it with 'use_nnapi=1' option.

TF-Lite Benchmark:-

adb shell /data/local/tmp/benchmark_model --graph=/data/local/tmp/deeplabv3_257_mv_gpu.tflite --num_threads=1 --use_nnapi=1 adb: /opt/intel/intelpython27/lib/libcrypto.so.1.0.0: no version information available (required by adb) STARTING! Min num runs: [50] Min runs duration (seconds): [1] Inter-run delay (seconds): [-1] Num threads: [1] Benchmark name: [] Output prefix: [] Min warmup runs: [1] Min warmup runs duration (seconds): [0.5] Graph: [/data/local/tmp/deeplabv3_257_mv_gpu.tflite] Input layers: [] Input shapes: [] Use nnapi : [1] Allow fp16 : [0] Loaded model /data/local/tmp/deeplabv3_257_mv_gpu.tflite resolved reporter Initialized session in 325.573ms Running benchmark for at least 1 iterations and at least 0.5 seconds Op code 23 is currently not delegated to NNAPI Returning error since TFLite returned failure nnapi_delegate.cc:736. Failed to build graph for NNAPI Failed to invoke! Aborted

Android Log:

android.example.com.tflitecamerademo E/tflite: Op code 23 is currently not delegated to NNAPI Returning error since TFLite returned failure nnapi_delegate.cc:751. Failed to build graph for NNAPI

[jdduke]

Thanks for flagging. We'll soon be moving to an NNAPI acceleration approach which allows partial subgraph delegation, which should resolve the problem.

[freedomtan]

@jdduke Op code 23 is ResizeBilinear (kTfLiteBuiltinResizeBilinear = 23), which is supported by NNAPI. That is, this is not an op in TFLite but not NNAPI.

[anilsathyan7]

Also, in the official tflite gpu classification demo NNAPI was not giving any speed up (even compared with CPU version) with float model.With quantized model CPU was having better speed than NNAPI. Benchmark result from app: CPU: ~ 200 ms GPU: ~ 90 ms NNAPI: ~ 300-500 ms

[freedomtan]

@anilsathyan7 it seems you are using NNAPI cpu fallback, that is, your device doesn't have real NNAPI accelerators.

[freedomtan]

@jdduke and @anilsathyan7 it seems there is a stalled PR try to delegate ResizeBilinear to NNAPI https://github.com/tensorflow/tensorflow/pull/19928

[anilsathyan7]

We also tried with Huawei Honor Play (Kirin 970 + NPU) with Android 9.0. The tflite demo app gives least performance, with both quantized and float models, when we switch to NNAPI !!! (i.e NNAPI < CPU < GPU).

[jdduke]

@miaowang14 I believe the partial delegation should be addressed in the latest Android/TFLite release?

[miaowang14]

This should have been fixed. Please re-open if that is not the case.

[google-ml-butler[bot]]

Are you satisfied with the resolution of your issue? Yes No