Object detection Batch inference : cloud ml engine built in docker image issue.

[Aashish-1008]

Hello,

I am using cloud ml engine built-in docker image for object detection training "gcr.io/cloud-ml-algos/image_object_detection:latest" Training and inference (for a single image) are working fine in the cloud ml engine, there is no issue.

But when I am trying to do batch inference for more than one image at a time, model is inferring the same output for all the input images. I am using cloud ml engine for inference:

Model deployed in cloud ml engine with this config:

"machine_type": "mls1-c1-m2",
"python_version": "3.5",
"runtime_version": "1.14",

request.json

{"encoded_image": {"b64": "/9j/4AAQSkZJRgABAQAAAQABAAD/..............", "key": "1"}}
{"encoded_image": {"b64": "/4j/QABAADQSkZJRQSkZJR/..............", "key": "2"}}

command to make inference request to ml engine:

gcloud ai-platform predict --model $MODEL_NAME \
 --version $VERSION_NAME \
 --json-instances request.json 

let's assume inference for the first image whose key:1 is "INFERENCE_RESULT_1"

MODEL INFERENCE RESULT::

"INFERENCE_RESULT_1"
"INFERENCE_RESULT_1"

These two input images used for prediction are different and output different predictions when I am making inference requests separately, but while making the request in the batch (of more than one) model is predicting the same output for all the images.

Thanks in advance.

[pkulzc]

Are you using Tensorflow Object Detection API to train your model?

[Aashish-1008]

I am using google ai platform for training which uses a docker image "gcr.io/cloud-ml-algos/image_object_detection:latest" stored in google container registry (GCR). Please check it out below link, I am using this guide for training, model deployment, and inference.

https://cloud.google.com/ai-platform/training/docs/algorithms/object-detection