Tensorflow Serving returns "Tensor name: probability has no shape information"

[pokonski]

For given exported model:

MetaGraphDef with tag-set: 'serve' contains the following SignatureDefs:

signature_def['serving_default']:
  The given SavedModel SignatureDef contains the following input(s):
    inputs['input'] tensor_info:
        dtype: DT_STRING
        shape: unknown_rank
        name: input_image_as_bytes:0
  The given SavedModel SignatureDef contains the following output(s):
    outputs['output'] tensor_info:
        dtype: DT_STRING
        shape: unknown_rank
        name: prediction:0
    outputs['probability'] tensor_info:
        dtype: DT_DOUBLE
        shape: unknown_rank
        name: probability:0
  Method name is: tensorflow/serving/predict

Sending a following request to the serving REST API:

curl -X POST \
  http://localhost:9001/v1/models/testmodel:predict \
  -H 'cache-control: no-cache' \
  -H 'content-type: application/json' \
  -d '{
  "signature_name": "serving_default",
  "instances": [
     { "b64": "/9j/4AAk=" }
   ]
}'

returns an error:

{
    "error": "Tensor name: probability has no shape information "
}

is it something that needs to be adjusted in the export function in aocr?

[emedvedev]

Oh, that's interesting. I'll try to get to it during the weekend.

[mz8i]

@emedvedev have you figured out what the problem was? I'm asking because I get the same error with a completely unrelated tensorflow model and this is the only place where I have found the same error message being mentioned :)

[xvonabur]

@emedvedev @pokonski @mz8i Did you find a solution? I faced exact same problem.

[pokonski]

Hey @xvonabur, I just wrote a simple API in python myself instead of using Serving.

[xvonabur]

@pokonski Can you open source it? Or give me some hints how to implement it myself? Unfortunately I do not know python at all, so I'm stuck for now.

[pokonski]

@xvonabur sure, I'll try to prepare an open-sourcable version since it currently contains work-related code.

[pokonski]

@xvonabur I've prepared a simple working server here: https://github.com/pokonski/attention-ocr-api

I expect this might come in handy for more people since this issue is the first result in Google when searching for this error :)

WARNING This is not a production-ready code, Serving would be ideal but until it's solved by smarter people I am using the Python server.

[xvonabur]

@pokonski Thanks! I've managed to run it on Debian. I would like to use it in production, but saw your warning and now I'm curious what drawbacks has your solution?

[pokonski]

It starts in a development mode by default which is slower and it probably doesn't handle thousands of requests well, but I guess that depends on the traffic you expect :)

See: https://vsupalov.com/flask-web-server-in-production/

[nathan-standafer]

I get the error as well. I am using the tensorflow/serving docker image. Has anybody found a way to export the model that is compatible with tensorflow/serving?

{ "error": "Tensor name: probability has no shape information " }

[MariusMez]

Hi,

Since months I'm stuck on the same problem using the curl command provided as exemple. But today I tried to understand why this error { "error": "Tensor name: probability has no shape information " } occur with the documentation here: https://www.tensorflow.org/serving/api_rest and I found an answer using this curl command instead:

curl -X POST --output - \
  http://yourserver:9001/v1/models/aocr:predict \
  -H 'cache-control: no-cache' \
  -H 'content-type: application/json' \
  -d '{
  "signature_name": "serving_default",
  "inputs": 
     {
        "input": { "b64": "/9j/4AAQSkZJRgABAQAASABIAAD/4QBYRXhpZgAATU0AKgAAAAgAAgESAAMAAAABAAEAAIdpAAQAAAABAAAAJgAAAAAAA6ABAAMAAAABAAEAAKACAAQAAAABAAAAOaADAAQAAAABAAAAHAAAAAD/7QA4UGhvdG9zaG9wIDMuMAA4QklNBAQAAAAAAAA4QklNBCUAAAAAABDUHYzZjwCyBOmACZjs+EJ+/8AAEQgAHAA5AwEiAAIRAQMRAf/EAB8AAAEFAQEBAQEBAAAAAAAAAAABAgMEBQYHCAkKC//EALUQAAIBAwMCBAMFBQQEAAABfQECAwAEEQUSITFBBhNRYQcicRQygZGhCCNCscEVUtHwJDNicoIJChYXGBkaJSYnKCkqNDU2Nzg5OkNERUZHSElKU1RVVldYWVpjZGVmZ2hpanN0dXZ3eHl6g4SFhoeIiYqSk5SVlpeYmZqio6Slpqeoqaqys7S1tre4ubrCw8TFxsfIycrS09TV1tfY2drh4uPk5ebn6Onq8fLz9PX29/j5+v/EAB8BAAMBAQEBAQEBAQEAAAAAAAABAgMEBQYHCAkKC//EALURAAIBAgQEAwQHBQQEAAECdwABAgMRBAUhMQYSQVEHYXETIjKBCBRCkaGxwQkjM1LwFWJy0QoWJDThJfEXGBkaJicoKSo1Njc4OTpDREVGR0hJSlNUVVZXWFlaY2RlZmdoaWpzdHV2d3h5eoKDhIWGh4iJipKTlJWWl5iZmqKjpKWmp6ipqrKztLW2t7i5usLDxMXGx8jJytLT1NXW19jZ2uLj5OXm5+jp6vLz9PX29/j5+v/bAEMACAYGBwYFCAcHBwkJCAoMFA0MCwsMGRITDxQdGh8eHRocHCAkLicgIiwjHBwoNyksMDE0NDQfJzk9ODI8LjM0Mv/bAEMBCQkJDAsMGA0NGDIhHCEyMjIyMjIyMjIyMjIyMjIyMjIyMjIyMjIyMjIyMjIyMjIyMjIyMjIyMjIyMjIyMjIyMv/dAAQABP/aAAwDAQACEQMRAD8AyYYwgG6rIIAHBqCLHFQeVLHP57SHPmhdmeApOK5bHKaKnjjjtzT1ljjLhnXKj5gD0rMaVxdtCsch2zA7ieADVqK1X7VMxX5y2Dk9QQD/AFq0hWLkc0fmhFbJKhvwNTxMHGQehway9LVUnl9D9xic/KKuW88e+ZVJyGLdO2KZJahZZYhIDgH9Kk2j+8KpWcu7chRhhmIJHByc1fwP8igR/9DKhIwNtI1oHuPM3tsyG8vtkd6bExAFW0Y4rmRx3AwD5gR94hjz3FXLqMJcOP4gFB+oAH9KhjG48+lWLg7rxye8hqhNkSRhQqqoAAwKsKVHtxjIHvUP3nGalQYFMBwOCD/+qpd/1pgUcU/NAH//2Q==" }
     }
}'

Yeah! I've the correct output:

{
    "outputs": {
        "probability": 0.999405,
        "output": "TTXJHC"
    }
}

I dont' know why we have to specify the input tensors in column format instead of using the row format.

Specifying input tensors in row format.

This format is similar to PredictRequest proto of gRPC API and the CMLE predict API. Use this format if all named input tensors have the same 0-th dimension. If they don't, use the columnar format described later below.

"instances": [ { "b64": "/9j/4AAk==" } ] --> { "error": "Tensor name: probability has no shape information " }

Specifying input tensors in column format.

Use this format to specify your input tensors, if individual named inputs do not have the same 0-th dimension or you want a more compact representation. This format is similar to the inputs field of the gRPC Predict request.

"inputs": { "input": { "b64": "/9j/4AAk==" } } --> Working

[emedvedev]

@MariusMez oh, that's amazing! Thank you so much, I didn't have time to actually get around to this issue. Would you mind updating the README? It would be much appreciated :)

[pokonski]

That is awesome :heart: @MariusMez