In TFnighty TFLite Interpreter missing setUseNNAPI "wrapper" method to call Interpreter.Options.setUseNNAPI

[grewe]

Please make sure that this is a bug. As per our GitHub Policy, we only address code/doc bugs, performance issues, feature requests and build/installation issues on GitHub. tag:bug_template

System information

• Have I written custom code (as opposed to using a stock example script provided in TensorFlow):
• OS Platform and Distribution (e.g., Linux Ubuntu 16.04): WIndows 10
• Mobile device (e.g. iPhone 8, Pixel 2, Samsung Galaxy) if the issue happens on mobile device: Android emulator
• TensorFlow installed from (source or binary): org.tensorflow:tensorflow-lite-gpu:0.0.0-nightly'
• TensorFlow version (use command below): nightly
• Python version:
• Bazel version (if compiling from source):
• GCC/Compiler version (if compiling from source):
• CUDA/cuDNN version:
• GPU model and memory:

You can collect some of this information using our environment capture script You can also obtain the TensorFlow version with:

1. TF 1.0: python -c "import tensorflow as tf; print(tf.GIT_VERSION, tf.VERSION)"
2. TF 2.0: python -c "import tensorflow as tf; print(tf.version.GIT_VERSION, tf.version.VERSION)"

Describe the current behavior Making call to Interpreter's setUseNNAPI method which no longer exists in the tfnightly version as of Oct. 12. The method it should call Interpreter.Options.setUseNNAPI still exists but, not in the Interperter itself.

Describe the expected behavior Either change the documentation to reflect this method does not exist or fix this.

Standalone code to reproduce the issue Provide a reproducible test case that is the bare minimum necessary to generate the problem. If possible, please share a link to Colab/Jupyter/any notebook.

tfLite = new Interpreter(loadModelFile(assetManager, modelFilename));

if (tfLite != null) tfLite.setUseNNAPI(isChecked);

Other info / logs Include any logs or source code that would be helpful to diagnose the problem. If including tracebacks, please include the full traceback. Large logs and files should be attached.

[freedomtan]

Interpreter.Options is a class inside Interpreter. It was implemented back in 2018. To use it, you have to create an instance of Interpreter.Options. That is, for your code, it should be something like:

Interpreter.Options tfliteOptions = new Interpreter.Options();
tfliteOptions.setUseNNAPI(isChecked);
tfLite = new Interpreter(loadModelFile(assetManager, modelFilename), tfliteOptions);

https://github.com/tensorflow/tensorflow/blob/master/tensorflow/lite/java/src/main/java/org/tensorflow/lite/Interpreter.java#L82-L183

[grewe]

I already know that Intrepreter.Options has this method and that is a work around to set the options in the constructer. My issue report was that prior Oct. 12 (or there abouts) like the setNumThreads it was implemented in BOTH the Interpreter class and the Interpreter.Options. This allowed the modification of the Interpreter object after construction. Also, note: this is performed by the sample in the new TF2 Object Detection API mobile app as they have a GUI element that can toggle ON/OFF the use of the NNAPI for acceleration and I believe others will complain.

[grewe]

I understand now from the documentation you linked to it is removing of a deprecated method however, given this I suggest that the TF2 Object Detection sample provided be reworked and the option removed or altered to recreate with GUI event each time the Interpreter. Otherwise, that team will get reports on broken sample code.

[freedomtan]

@grewe Got your point. I don't work for Google, just happened to use Interpreter.Options before. Tag Jared @jdduke who removed Interpreter.setUseNNAPI method in https://github.com/tensorflow/tensorflow/commit/89c2f7f062ee125bd3783035b1be07b5c0f9ea73 according to git log.

[jdduke]

The object detection sample has already been updated to use the Interpreter.Options variant. It may not be as convenient, but since 2019, using Interpreter.setUseNNAPI had a few nasty corner cases (e.g., you couldn't disable it once it has been enabled and used, and the first inference result is almost always incorrect).

We'll also likely be removing the Interpreter.setNumThreads() setter as well. It's note quite as flawed, but it does have some unfortunate corner cases as well.

[google-ml-butler[bot]]

Are you satisfied with the resolution of your issue? Yes No

[DESUP2]

If we don't use Interpreter.setNumThreads() as deprecated already then Interpreter.Options automatically collects data ? or new alternative method arrived.