Cannot get Mobilenet V3 SSD to produce useful output

idenc commented 5 years ago

Please go to Stack Overflow for help and support:

http://stackoverflow.com/questions/tagged/tensorflow

Also, please understand that many of the models included in this repository are experimental and research-style code. If you open a GitHub issue, here is our policy:

It must be a bug, a feature request, or a significant problem with documentation (for small docs fixes please send a PR instead).
The form below must be filled out.

Here's why we have that policy: TensorFlow developers respond to issues. We want to focus on work that benefits the whole community, e.g., fixing bugs and adding features. Support only helps individuals. GitHub also notifies thousands of people when issues are filed. We want them to see you communicating an interesting problem, rather than being redirected to Stack Overflow.

System information

What is the top-level directory of the model you are using: Mobilenet V3 SSD small/large downloaded from https://github.com/tensorflow/models/blob/master/research/object_detection/g3doc/detection_model_zoo.md
Have I written custom code (as opposed to using a stock example script provided in TensorFlow): Yes, have also tried stock example scripts
OS Platform and Distribution (e.g., Linux Ubuntu 16.04): Windows 10 Pro 1903
TensorFlow installed from (source or binary): From pip binary
TensorFlow version (use command below): Tensorflow 2.0
Bazel version (if compiling from source):
CUDA/cuDNN version: NA
GPU model and memory: NA using CPU
Exact command to reproduce: NA not from a command

Describe the problem

No matter what I try it seems the Mobilenet V3 SSD outputs all zeroes. I downloaded both the small and large models from the detection model zoo. I have tried using multiple images, both normalized and unnormalized, running both the provided .tflite and frozen .pbs for the small and large models, and using different installs of Tensorflow to no avail. I also tried using the object_detection_tutorial.ipynb provided but MBV3 SSD does not work with that script. What am I doing wrong? Or is there a problem with the provided model?

Source code / logs

Include any logs or source code that would be helpful to diagnose the problem. If including tracebacks, please include the full traceback. Large logs and files should be attached. Try to provide a reproducible test case that is the bare minimum necessary to generate the problem.


tf.compat.v1.disable_v2_behavior()
import numpy as np
import cv2

def load_graph(frozen_graph_filename):
    # We load the protobuf file from the disk and parse it to retrieve the
    # unserialized graph_def
    with tf.io.gfile.GFile(frozen_graph_filename, "rb") as f:
        graph_def = tf.compat.v1.GraphDef()
        graph_def.ParseFromString(f.read())

    # Then, we import the graph_def into a new Graph and returns it
    with tf.Graph().as_default() as graph:
        # The name var will prefix every op/nodes in your graph
        # Since we load everything in a new graph, this is not needed
        tf.import_graph_def(graph_def, name="")
    return graph

model_graph = load_graph(r"C:\Users\iden\Downloads\ssd_mobilenet_v3_small_coco_2019_08_14\frozen_inference_graph.pb")
outputs = [model_graph.get_tensor_by_name("raw_outputs/class_predictions:0"),
           model_graph.get_tensor_by_name("raw_outputs/box_encodings:0")]

image = cv2.imread(r"C:\Users\iden\Pictures\test.png").astype(np.float32)
image = cv2.resize(image, (320, 320))
image = cv2.cvtColor(image, cv2.COLOR_BGR2RGB)
image = (image - 128) / 127.5
image = np.expand_dims(image, 0)

with tf.compat.v1.Session(graph=model_graph) as sess:
    output_tensors = sess.run(outputs, feed_dict={'normalized_input_image_tensor:0': image})

scores = output_tensors[0]
print(scores)
locations = output_tensors[1]
print(locations)

This script gives output:

[[[0.5 0.5 0.5 ... 0.5 0.5 0.5]
  [0.5 0.5 0.5 ... 0.5 0.5 0.5]
  [0.5 0.5 0.5 ... 0.5 0.5 0.5]
  ...
  [0.5 0.5 0.5 ... 0.5 0.5 0.5]
  [0.5 0.5 0.5 ... 0.5 0.5 0.5]
  [0.5 0.5 0.5 ... 0.5 0.5 0.5]]]
[[[-1.2356030e-17  1.4101090e-17 -6.3667132e-18  3.1901107e-17]
  [ 7.7017563e-18  4.9004532e-17 -7.8113352e-17 -9.1179292e-19]
  [-6.6128603e-17  3.8230567e-17 -7.8168674e-17 -3.8976880e-17]
  ...
  [ 7.6168566e-32  2.0741278e-30 -1.5562906e-30  1.0528488e-30]
  [-5.0384362e-30 -1.5268065e-30 -2.2974830e-30  1.3904762e-30]
  [-2.2760488e-30 -1.1501334e-30 -2.2292421e-30  1.1195719e-31]]]

So the input to the sigmoid is all zero and the box outputs are all basically zero (I have checked the whole tensors for larger values).

elrid commented 5 years ago

I'm having the same issue on iOS object_detection sample app.

I'm not sure you are correctly applying quantisation on input, input is in uint8, and zero point is 127, and I believe that network expects components to be in 127-255 range, thus image transform should look like image = image / 2.0 + 127.0 if components in cv2 are ranged from 0 to 255. If they are ranged from 0 to 1, image transform should be image = image * 128.0 + 127.0

However, both conversions still result in zero'ed tensor outputs

haoyuantan commented 5 years ago

How do you know the input(feed_dict) and the outputs?

elrid commented 4 years ago

Here is working tflite small model with refrozen pb from provided checkpoint. ssd_mobilenet_v3_coco.zip

filipvg68 commented 4 years ago

@elrid : works very well! Thank you for posting.

filipvg68 commented 4 years ago

@elrid: could you outline how you have re-created the tflite model?

UsedToBe97 commented 4 years ago

@elrid Hi, elrid, could you please provide a tflite with only uint8 computation?

thias15 commented 4 years ago

@elrid: How did you do the conversion? I would like to do something similar for other detector models from the model zoo.

angyee commented 4 years ago

Here is working tflite small model with refrozen pb from provided checkpoint. ssd_mobilenet_v3_coco.zip

where is the inference file to run this model? share a link? share link for this resource all?

Shoshin23 commented 4 years ago

@elrid I'm using the standard tflite converter and facing issues with Mobilnet V3's inference graph file. It's not returning any output at all. Can you tell me how you got it to work for you? Any script you use to do the conversion?

Thanks!

idenc commented 4 years ago

this pull request has links to correct model checkpoints with inference files.

Syirrus commented 4 years ago

Where can I find the label.txt file to this dataset? I have no idea where I can find it.

Ammu1991 commented 4 years ago

Where can I find the label.txt file to this dataset? I have no idea where I can find it.

You can use label.txt file provided with demo example code. It is working for me with v3 model.

Ammu1991 commented 4 years ago

Here is working tflite small model with refrozen pb from provided checkpoint. ssd_mobilenet_v3_coco.zip

Thank you so much for the help...cheers!!!

Ammu1991 commented 4 years ago

Here is working tflite small model with refrozen pb from provided checkpoint. ssd_mobilenet_v3_coco.zip

@elrid Do you have any idea similar to your recreated model do we have officially released v3 model by tensorflow on their website?

sumeshpandit commented 4 years ago

@elrid @filipvg68 @Ammu1991 can you tell me the output and input dimensoins of variables(i.e images and outputmap) in your refrozen ssd_mobilenet_v3 model

Lotfi-MERAD commented 4 years ago

Here is working tflite small model with refrozen pb from provided checkpoint. ssd_mobilenet_v3_coco.zip

it is mobilnet-v2 not v3 ;)

angyee commented 4 years ago

Here is working tflite small model with refrozen pb from provided checkpoint. ssd_mobilenet_v3_coco.zip

it is mobilnet-v2 not v3 ;)

How? proof?

animeesh commented 2 years ago

still not getting any way to run ssd mobilenet v3 ? if anyone have the resources do attached the link.

tensorflow / models