search

PINTO0309 / PINTO_model_zoo

A repository for storing models that have been inter-converted between various frameworks. Supported frameworks are TensorFlow, PyTorch, ONNX, OpenVINO, TFJS, TFTRT, TensorFlowLite (Float32/16/INT8), EdgeTPU, CoreML.
https://qiita.com/PINTO
MIT License
3.49k stars 566 forks source link

Error in image segmentation tflite model running on android? #329

Closed t109368507 closed 1 year ago

t109368507 commented 1 year ago

Issue Type

Support

OS

Ubuntu

OS architecture

aarch64

Programming Language

Python

Framework

TensorFlowLite

Model name and Weights/Checkpoints URL

Deeplabv3 model

Description

According to the Image Segmentation tutorial, the following error message appears when running on Android. How to solve it? Thanks!

I/tflite: Replacing 127 node(s) with delegate (TfLiteXNNPackDelegate) node, yielding 29 partitions.
E/TaskJniUtils: Error getting native address of native library: task_vision_jni
    java.lang.RuntimeException: Error occurred when initializing ImageSegmenter: Input tensor has type kTfLiteFloat32: it requires specifying NormalizationOptions metadata to preprocess input images.
        at org.tensorflow.lite.task.vision.segmenter.ImageSegmenter.initJniWithModelFdAndOptions(Native Method)
        at org.tensorflow.lite.task.vision.segmenter.ImageSegmenter.access$100(ImageSegmenter.java:77)
        at org.tensorflow.lite.task.vision.segmenter.ImageSegmenter$3.createHandle(ImageSegmenter.java:419)
        at org.tensorflow.lite.task.core.TaskJniUtils.createHandleFromLibrary(TaskJniUtils.java:91)
        at org.tensorflow.lite.task.vision.segmenter.ImageSegmenter.createFromModelFdAndOptions(ImageSegmenter.java:415)
        at org.tensorflow.lite.task.vision.segmenter.ImageSegmenter.createFromFileAndOptions(ImageSegmenter.java:139)
        at org.tensorflow.lite.examples.imagesegmentation.ImageSegmentationHelper.setupImageSegmenter(ImageSegmentationHelper.kt:99)
        at org.tensorflow.lite.examples.imagesegmentation.ImageSegmentationHelper.segment(ImageSegmentationHelper.kt:116)
        at org.tensorflow.lite.examples.imagesegmentation.fragments.CameraFragment.segmentImage(CameraFragment.kt:265)
        at org.tensorflow.lite.examples.imagesegmentation.fragments.CameraFragment.bindCameraUseCases$lambda-6$lambda-5(CameraFragment.kt:240)
        at org.tensorflow.lite.examples.imagesegmentation.fragments.CameraFragment.$r8$lambda$XZey5CsMADmZQ6yO-iU2A2TMxzQ(Unknown Source:0)
        at org.tensorflow.lite.examples.imagesegmentation.fragments.CameraFragment$$ExternalSyntheticLambda0.analyze(Unknown Source:2)
        at androidx.camera.core.ImageAnalysis.lambda$setAnalyzer$2(ImageAnalysis.java:477)
        at androidx.camera.core.ImageAnalysis$$ExternalSyntheticLambda2.analyze(Unknown Source:2)
        at androidx.camera.core.ImageAnalysisAbstractAnalyzer.lambda$analyzeImage$0$androidx-camera-core-ImageAnalysisAbstractAnalyzer(ImageAnalysisAbstractAnalyzer.java:285)
        at androidx.camera.core.ImageAnalysisAbstractAnalyzer$$ExternalSyntheticLambda1.run(Unknown Source:14)
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1167)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:641)
        at java.lang.Thread.run(Thread.java:919)

Relevant Log Output

I/tflite: Replacing 127 node(s) with delegate (TfLiteXNNPackDelegate) node, yielding 29 partitions.
E/TaskJniUtils: Error getting native address of native library: task_vision_jni
    java.lang.RuntimeException: Error occurred when initializing ImageSegmenter: Input tensor has type kTfLiteFloat32: it requires specifying NormalizationOptions metadata to preprocess input images.
        at org.tensorflow.lite.task.vision.segmenter.ImageSegmenter.initJniWithModelFdAndOptions(Native Method)
        at org.tensorflow.lite.task.vision.segmenter.ImageSegmenter.access$100(ImageSegmenter.java:77)
        at org.tensorflow.lite.task.vision.segmenter.ImageSegmenter$3.createHandle(ImageSegmenter.java:419)
        at org.tensorflow.lite.task.core.TaskJniUtils.createHandleFromLibrary(TaskJniUtils.java:91)
        at org.tensorflow.lite.task.vision.segmenter.ImageSegmenter.createFromModelFdAndOptions(ImageSegmenter.java:415)
        at org.tensorflow.lite.task.vision.segmenter.ImageSegmenter.createFromFileAndOptions(ImageSegmenter.java:139)
        at org.tensorflow.lite.examples.imagesegmentation.ImageSegmentationHelper.setupImageSegmenter(ImageSegmentationHelper.kt:99)
        at org.tensorflow.lite.examples.imagesegmentation.ImageSegmentationHelper.segment(ImageSegmentationHelper.kt:116)
        at org.tensorflow.lite.examples.imagesegmentation.fragments.CameraFragment.segmentImage(CameraFragment.kt:265)
        at org.tensorflow.lite.examples.imagesegmentation.fragments.CameraFragment.bindCameraUseCases$lambda-6$lambda-5(CameraFragment.kt:240)
        at org.tensorflow.lite.examples.imagesegmentation.fragments.CameraFragment.$r8$lambda$XZey5CsMADmZQ6yO-iU2A2TMxzQ(Unknown Source:0)
        at org.tensorflow.lite.examples.imagesegmentation.fragments.CameraFragment$$ExternalSyntheticLambda0.analyze(Unknown Source:2)
        at androidx.camera.core.ImageAnalysis.lambda$setAnalyzer$2(ImageAnalysis.java:477)
        at androidx.camera.core.ImageAnalysis$$ExternalSyntheticLambda2.analyze(Unknown Source:2)
        at androidx.camera.core.ImageAnalysisAbstractAnalyzer.lambda$analyzeImage$0$androidx-camera-core-ImageAnalysisAbstractAnalyzer(ImageAnalysisAbstractAnalyzer.java:285)
        at androidx.camera.core.ImageAnalysisAbstractAnalyzer$$ExternalSyntheticLambda1.run(Unknown Source:14)
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1167)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:641)
        at java.lang.Thread.run(Thread.java:919)

URL or source code for simple inference testing code

Script of convert tflite

import os
import numpy as np
import tensorflow as tf

print(tf.__version__)

MODEL_FILE = "frozen_inference_graph_257.pb"

converter = tf.compat.v1.lite.TFLiteConverter.from_frozen_graph(
    graph_def_file = MODEL_FILE,
    input_arrays = ['sub_2'], # For the Xception model it needs to be `sub_7`, for MobileNet it would be `sub_2`
    output_arrays = ['ResizeBilinear_2'],
    input_shapes={'sub_2':[1,257,257,3]}
)

converter.optimizations = [tf.lite.Optimize.DEFAULT]

tflite_model = converter.convert()

tflite_path = "deeplabv3_mnv2_custom_257.tflite"
with open(tflite_path, 'wb') as f:
    f.write(tflite_model)
t109368507 commented 1 year ago

Is quantization not correct? image

PINTO0309 commented 1 year ago

Questions about implementation should be asked here. https://discuss.tensorflow.org/