Tflite converted file output weird result

[hamhochoi]

Hi all,

As the title, I got strange results when inference using tflite converted weight file.

OS: Windows 10 tensorflow 2.3.1

Converted command:
python model_inspect.py --runmode=saved_model --model_name=efficientdet-d0 --ckpt_path=efficientdet-d0 --saved_model_dir=savedmodeldir --tflite_path=efficientdet-d0.tflite --hparams="image_size=1920x1280"

Result using .pb file: Command:

python model_inspect.py --runmode=saved_model_infer   --saved_model_dir="savedmodeldir" --model_name="efficientdet-d0"  --input_image=testdata/2.jpg   --output_image_dir="serve_image_out"  --min_score_thresh=0.35  --max_boxes_to_draw=200

Result using tflite file: File test: https://drive.google.com/file/d/12zXA51bVYnKHCvuue2-uClbZCISTZ_eY/view?usp=sharing File tflite: https://drive.google.com/file/d/1aLDdZYmmraPIAHc5OHPQ0QnG6tk9dBB9/view?usp=sharing

Any idea where I got wrong?

Many thanks,

[hamadichihaoui]

@hamhochoi the tflite model is float 32 bit or 8 bit?

[hamhochoi]

@hamhochoi the tflite model is float 32 bit or 8 bit?

As you can download my tflite model in link above, the input is uint8, output is float32

[mingxingtan]

@hamadichihaoui How did you run inference with the tflite? You might miss some pre-processing or post-processing?

[hamhochoi]

@hamadichihaoui How did you run inference with the tflite? You might miss some pre-processing or post-processing?

Thank you for replying me, I run inference by using the code here

[Steve-2040]

You can also have a look at this https://github.com/google/automl/issues/753#issuecomment-689353891 if it is a saved_model. (I have not used tflite myself) For the saved_model you get:

saved_model_cli show --dir savedmodeldir --all

signature_def['serving_default']:
  The given SavedModel SignatureDef contains the following input(s):
    inputs['image_arrays:0'] tensor_info:
        dtype: DT_UINT8
        shape: (-1, -1, -1, -1)
        name: image_arrays:0
  The given SavedModel SignatureDef contains the following output(s):
    outputs['detections:0'] tensor_info:
        dtype: DT_FLOAT
        shape: (1, 100, 7)
        name: detections:0
  Method name is: tensorflow/serving/predict

[fsx950223]

input_h = 1920
input_w = 1280
input_tensor = img.resize((input_h, input_w))

You just resize the image but don't scale the bboxs back, so you get a weird result.

[hamhochoi]

bboxs back, so you get a weird result.

Thanks for your comment but actually I draw bbox on the resized image not the original image so the bbox shouldn't be scale back. So I still don't know why the result is weird.

img = Image.fromarray(input_tensor[0])
draw = ImageDraw.Draw(img)
for i in range(len(ymin)):
    draw.rectangle([(xmin[i], ymin[i]), (xmax[i], ymax[i])], outline='red')
img.show()
img.save("result.jpg")

[fsx950223]

bboxs back, so you get a weird result.

Thanks for your comment but actually I draw bbox on the resized image not the original image so the bbox shouldn't be scale back. So I still don't know why the result is weird.

img = Image.fromarray(input_tensor[0])
draw = ImageDraw.Draw(img)
for i in range(len(ymin)):
  draw.rectangle([(xmin[i], ymin[i]), (xmax[i], ymax[i])], outline='red')
img.show()
img.save("result.jpg")

from PIL import ImageDraw, ImageFont, Image
import cv2
import glob,os
import numpy as np
import matplotlib.pyplot as plt
import tensorflow as tf
import time

car_tflite = 'tflite_test/fp32.tflite'
interpreter = tf.lite.Interpreter(car_tflite)
interpreter.allocate_tensors()
input_details = interpreter.get_input_details()
output_details = interpreter.get_output_details()

img_input = 'testdata/img1.jpg'
img = Image.open(img_input)
image_w, image_h = img.size

input_h = 512
input_w = 512
input_tensor = img.resize((input_h, input_w))
input_tensor = np.array(input_tensor, dtype=np.uint8)
input_tensor = np.expand_dims(input_tensor,axis=0)
#input_tensor /= 255.0
interpreter.set_tensor(input_details[0]['index'], input_tensor)
start_time = time.process_time()
interpreter.invoke()
run_time = time.process_time() - start_time
print('Inference time for car:',run_time)

p1 = interpreter.get_tensor(output_details[0]['index'])
scores = interpreter.get_tensor(output_details[1]['index'])
labels = interpreter.get_tensor(output_details[2]['index'])
print (p1.shape)

# img_id = p1[..., 0]
ymin, xmin, ymax, xmax = p1[..., 0], p1[..., 1], p1[..., 2], p1[..., 3]
# scores  = p1[..., 5]
# labels  = p1[..., 6]

class_threshold = 0.5
idx = np.where(scores > class_threshold)
# img_id = img_id[idx]
ymin, xmin, ymax, xmax = ymin[idx], xmin[idx], ymax[idx], xmax[idx]
scores  = scores[idx]
labels  = labels[idx]

img = Image.fromarray(input_tensor[0])
draw = ImageDraw.Draw(img)
for i in range(len(ymin)):
    draw.rectangle([(xmin[i], ymin[i]), (xmax[i], ymax[i])], outline='red')
img.show()
img.save("result.jpg")

 python keras/inspector.py --model_name=efficientdet-d0 --mode=export --tflite=FP32 --model_dir=../../download/efficientdet-d0 --saved_model_dir=./tflite_test

Worked.

[hamhochoi]

bboxs back, so you get a weird result.

Thanks for your comment but actually I draw bbox on the resized image not the original image so the bbox shouldn't be scale back. So I still don't know why the result is weird.

img = Image.fromarray(input_tensor[0])
draw = ImageDraw.Draw(img)
for i in range(len(ymin)):
    draw.rectangle([(xmin[i], ymin[i]), (xmax[i], ymax[i])], outline='red')
img.show()
img.save("result.jpg")

from PIL import ImageDraw, ImageFont, Image
import cv2
import glob,os
import numpy as np
import matplotlib.pyplot as plt
import tensorflow as tf
import time

car_tflite = 'tflite_test/fp32.tflite'
interpreter = tf.lite.Interpreter(car_tflite)
interpreter.allocate_tensors()
input_details = interpreter.get_input_details()
output_details = interpreter.get_output_details()

img_input = 'testdata/img1.jpg'
img = Image.open(img_input)
image_w, image_h = img.size

input_h = 512
input_w = 512
input_tensor = img.resize((input_h, input_w))
input_tensor = np.array(input_tensor, dtype=np.uint8)
input_tensor = np.expand_dims(input_tensor,axis=0)
#input_tensor /= 255.0
interpreter.set_tensor(input_details[0]['index'], input_tensor)
start_time = time.process_time()
interpreter.invoke()
run_time = time.process_time() - start_time
print('Inference time for car:',run_time)

p1 = interpreter.get_tensor(output_details[0]['index'])
scores = interpreter.get_tensor(output_details[1]['index'])
labels = interpreter.get_tensor(output_details[2]['index'])
print (p1.shape)

# img_id = p1[..., 0]
ymin, xmin, ymax, xmax = p1[..., 0], p1[..., 1], p1[..., 2], p1[..., 3]
# scores  = p1[..., 5]
# labels  = p1[..., 6]

class_threshold = 0.5
idx = np.where(scores > class_threshold)
# img_id = img_id[idx]
ymin, xmin, ymax, xmax = ymin[idx], xmin[idx], ymax[idx], xmax[idx]
scores  = scores[idx]
labels  = labels[idx]

img = Image.fromarray(input_tensor[0])
draw = ImageDraw.Draw(img)
for i in range(len(ymin)):
  draw.rectangle([(xmin[i], ymin[i]), (xmax[i], ymax[i])], outline='red')
img.show()
img.save("result.jpg")

 python keras/inspector.py --model_name=efficientdet-d0 --mode=export --tflite=FP32 --model_dir=../../download/efficientdet-d0 --saved_model_dir=./tflite_test

Worked.

Yes, thank you. the problem is with bboxes extract from output tensors. I've just figured out it. Thanks you very much.