TensorFlow Serving AMI for AWS not working

[brujbaur]

Platform

AWS

bndiagnostic ID know more about bndiagnostic ID

na

bndiagnostic output

na

bndiagnostic was not useful. Could you please tell us why?

the suggested guides are not related

Describe your issue as much as you can

Followed the instructions perfectly multiple times. Continue to get this error when running the resnet_client_cc script. gRPC call return code: 14: failed to connect to all addresses gRPC failed. Followed another technical solution on github and got this error. gRPC call return code: 3: Signature input alias: image_bytes(feed name: input_tensor:0) not found in request and no default value provided. Inputs expected to be in the request {input_1}, or default input alias set: gRPC failed.

Tried loading alternative resnet models and changing the config file. same results. This AMI seems not to work.

[gongomgra]

Hi @brujbaur,

Thanks for reporting this issue. I have reproduced it and I have found a solution. There is a bug in the initialization logic on our side, and the location of the installed models is not properly configured on the server instance. Please run the next command to update the Tensorflow Serving configuration

sudo sed -i "s#/bitnami/model-data#/opt/bitnami/model-data#" /opt/bitnami/tensorflow-serving/conf/tensorflow-serving.conf

After that, you should be able to execute predictions on the instance

$ resnet_client_cc --image_file=cat.jpg
calling predict using file: cat.jpg  ...
call predict ok
outputs size is 1
the result tensor[0] is:
(...)
Done.

We will work on releasing a new image with the config file correctly pointing to the installed models.

Hope it helps!

[brujbaur]

This config file does not have a reference to model-data, therefore the sed command is doing nothing. Can you send me the full contents of what is supposed to be in the .conf file please?

On Mon, Sep 30, 2024 at 4:47 AM Gonzalo Gómez Gracia < @.***> wrote:

Hi @brujbaur https://github.com/brujbaur,

Thanks for reporting this issue. I have reproduced it and I have found a solution. There is a bug in the initialization logic on our side, and the location of the installed models is not properly configured on the server instance. Please run the next command to update the Tensorflow Serving configuration

sudo sed -i "s#/bitnami/model-data#/opt/bitnami/model-data#" /opt/bitnami/tensorflow-serving/conf/tensorflow-serving.conf

After that, you should be able to execute predictions on the instance

$ resnet_client_cc --image_file=cat.jpg calling predict using file: cat.jpg ... call predict ok outputs size is 1 the result tensor[0] is: (...) Done.

We will work on releasing a new image with the config file correctly pointing to the installed models.

Hope it helps!

— Reply to this email directly, view it on GitHub https://github.com/bitnami/vms/issues/1662#issuecomment-2382644120, or unsubscribe https://github.com/notifications/unsubscribe-auth/A3ZK5BZ2WOHLQFRGJZOETXDZZEMZ7AVCNFSM6AAAAABO2ODBAWVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDGOBSGY2DIMJSGA . You are receiving this because you were mentioned.Message ID: @.***>

-- Bruce Baur (816) 550-1465

[gongomgra]

@brujbaur can you tell us which AMI are you using? I used the most recent one from our website: "bitnami-tensorflow-serving-2.17.0-1-r01-linux-debian-12-x86_64-hvm-ebs-nami" in the us-east-1 region. Can you give it a try?

https://bitnami.com/stack/tensorflow-serving/cloud/aws/amis

This is the content of the updated config file

bitnami@ip-172-31-86-230:~$ cat /opt/bitnami/tensorflow-serving/conf/tensorflow-serving.conf
model_config_list: {
  config: {
    name: "resnet",
    base_path: "/opt/bitnami/model-data",
    model_platform: "tensorflow",
  }
}

[brujbaur]

Yes that is the exact image AMI we are using. We were able to edit the config file as per your most recent update example, however still getting the same error. Here is the error. gRPC call return code: 3: Signature input alias: image_bytes(feed name: input_tensor:0) not found in request and no default value provided. Inputs expected to be in the request {input_1}, or default input alias set: gRPC failed.

[brujbaur]

I ran the diagnostics tool and uploaded it for you. 357e3d91-d8f0-f23b-1564-fe283c27aa03