aws / sagemaker-tensorflow-serving-container

A TensorFlow Serving solution for use in SageMaker. This repo is now deprecated.
Apache License 2.0
174 stars 101 forks source link

Docs on how to submit an invoke endpoint request for images, comparison to predict method #223

Open rbavery opened 2 years ago

rbavery commented 2 years ago

What did you find confusing? Please describe. I tried to submit an invoke endpoint request to a serverless endpoint deployed with sagemaker that uses one of these containers. however I can only find docs describing how to submit a request using a predictor variable, like so

predictor = tensorflow_serving_model.deploy(initial_instance_count=1,
                                            framework_version='1.12',
                                            instance_type='ml.p2.xlarge')
prediction = predictor.predict(data)

as described here https://github.com/aws/sagemaker-tensorflow-serving-container#deploying-a-tensorflow-serving-model

Describe how documentation can be improved An equivalent way to submit a request for inferences using invoke endpoint, specifically for images.

Additional context

I have a way to do this for an old endpoint but it doesn't seem to work for newly deployed sagemaker endpoints (even though the same model.tar.gz is used for both)

This works but only for old endpoints and there are multiple issues on SO and github describing difficulty with this not working anymore but being described in sagemaker docs, example:

# invoke endpoint
response = client.invoke_endpoint(
    EndpointName = "mira-large",
    ContentType = 'application/x-image', 
    Body = buf.getvalue()
)

# parse response
response_body = response['Body'].read()
response_body = response_body.decode('utf-8')
pred = json.loads(response_body)["predictions"][0]

output = {}
for i in range(len(pred)): output[CLASSES[i]] = float(pred[i])

print(output)
rbavery commented 2 years ago

Here is an official example that shows the application-x-image method that doesn't work: https://docs.aws.amazon.com/sagemaker/latest/dg/neo-requests-boto3.html

I get this error

ModelError: An error occurred (ModelError) when calling the InvokeEndpoint operation: Received client error (415) from model with message "{"error": "Unsupported Media Type: application/x-image"}"