Cannot use tf.saved_model.load() on a Mask RCNN model exported with exporter_main_v2

[zshancock]

Prerequisites

Please answer the following questions for yourself before submitting an issue.

• [x] I am using the latest TensorFlow Model Garden release and TensorFlow 2.
• [x] I am reporting the issue to the correct repository. (Model Garden official or research directory)
• [x] I checked to make sure that this issue has not already been filed.

1. The entire URL of the file you are using

https://github.com/tensorflow/models/blob/master/research/object_detection/exporter_main_v2.py

2. Describe the bug

I trained a model with model_main_tf2.py then used exporter_main_v2.py (with encoded_image_string_tensor) on the checkpoint directory and that generated the expected files (chiefly the saved_model.pb). However, when I try to load the model with tf.saved_model.load() I get unexpected behavior. I cannot seem to figure out how to make the call to the loaded model as it has 200+ inputs of Unknown name, and the outputs do not look correct either. I looked everywhere for signatures but can only find a concrete function.

3. Steps to reproduce

• The model closest to my implementation is the mask_rcnn_inception_resnet_tf2.config
• train model with model_main_tf2.py
• on the checkpoint directory from training, run exporter_main_v2.py with encoded_image_string_tensor as the input_type.
• attempt to load the resulting saved_model.pb with tf.saved_model.load()

4. Expected behavior

• Encode an image with inputs = base64.b64encode(img_data).decode() where img_data is a bytes-like representation
• load model with `model = tf.saved_model.load()
• get predictions with the call to model with model(inputs)

5. Additional context

I made several attempts to resolve - one error that came up often was ->

tensorflow.python.framework.errors_impl.InvalidArgumentError: assertion failed: [Unable to decode bytes as JPEG, PNG, or GIF]

(unexpected) The inputs of my loaded model with model.signatures['serving_default'].inputs :

[<tf.Tensor 'input_tensor:0' shape=(1,) dtype=string>, <tf.Tensor 'unknown:0' shape=<unknown> dtype=resource>, <tf.Tensor 'unknown_0:0' shape=<unknown> dtype=resource>, <tf.Tensor 'unknown_1:0' shape=<unknown> dtype=resource>, <tf.Tensor 'unknown_2:0' shape=<unknown> dtype=resource>, <tf.Tensor 'unknown_3:0' shape=<unknown> dtype=resource>, <tf.Tensor 'unknown_4:0' shape=<unknown> dtype=resource>,
 ...
 <tf.Tensor 'unknown_280:0' shape=<unknown> dtype=resource>]

(unexpected) The outputs with `model.signatures['serving_default'].outputs:

[<tf.Tensor 'Identity:0' shape=(None, 4) dtype=float32>, <tf.Tensor 'Identity_1:0' shape=(300, 4, 4, 2048) dtype=float32>, <tf.Tensor 'Identity_2:0' shape=(300, 121) dtype=float32>, <tf.Tensor 'Identity_3:0' shape=(1, 150) dtype=float32>, <tf.Tensor 'Identity_4:0' shape=(1, 150, 4) dtype=float32>, <tf.Tensor 'Identity_5:0' shape=(1, 150) dtype=float32>, <tf.Tensor 'Identity_6:0' shape=(1, 150, 15, 15) dtype=float32>, <tf.Tensor 'Identity_7:0' shape=(1, 150, 121) dtype=float32>, <tf.Tensor 'Identity_8:0' shape=(1, 150) dtype=float32>, <tf.Tensor 'Identity_9:0' shape=(1, 300, 4) dtype=float32>, <tf.Tensor 'Identity_10:0' shape=(4,) dtype=float32>, <tf.Tensor 'Identity_11:0' shape=(150, 120, 15, 15) dtype=float32>, <tf.Tensor 'Identity_12:0' shape=(1,) dtype=float32>, <tf.Tensor 'Identity_13:0' shape=(1,) dtype=float32>, <tf.Tensor 'Identity_14:0' shape=(1, 300, 4) dtype=float32>, <tf.Tensor 'Identity_15:0' shape=(1, 300, 4) dtype=float32>, <tf.Tensor 'Identity_16:0' shape=(1, 300, 4) dtype=float32>, <tf.Tensor 'Identity_17:0' shape=(1, 300, 121) dtype=float32>, <tf.Tensor 'Identity_18:0' shape=(300, 120, 4) dtype=float32>, <tf.Tensor 'Identity_19:0' shape=(1, 25600, 4) dtype=float32>, <tf.Tensor 'Identity_20:0' shape=(1, 25600, 2) dtype=float32>]

6. System information

I am using the tensorflow/tensorflow:2.3.0-gpu Docker Image

print(tf.version.GIT_VERSION, tf.version.VERSION)
v2.3.0-rc2-23-gb36436b087 2.3.0

[zshancock]

Additionally, I tried putting this model on AI Platform (framework 2.3) and making the call with this payload (worked for Mask RCNN of similar configuration in TF 1.15 framework.

{"instances": 
[
{"inputs": {"b64": encoded_image}
}
]
}

And that call error'd with:

"Failed to process element: 0 key: inputs of 'instances' list. Error: Invalid argument: JSON object: does not have named input: inputs"
}">

[daruai]

Additionally, I tried putting this model on AI Platform (framework 2.3) and making the call with this payload (worked for Mask RCNN of similar configuration in TF 1.15 framework.

{"instances": 
[
{"inputs": {"b64": encoded_image}
}
]
}

And that call error'd with:

"Failed to process element: 0 key: inputs of 'instances' list. Error: Invalid argument: JSON object: does not have named input: inputs"
}">

I have the same problem when I try to put the maskrcnn from the model zoo on AI Platform models for inference.