search

bentoml / aws-sagemaker-deploy

Fast model deployment on AWS Sagemaker
Apache License 2.0
15 stars 15 forks source link

Added support for `mulitpart/form-data` in AWS API Gateway #17

Closed jjmachan closed 3 years ago

jjmachan commented 3 years ago

This is supposed to fix https://github.com/bentoml/BentoML/issues/1822

jjmachan commented 3 years ago

/test-e2e-deploy sha=564aaa8

jjmachan commented 3 years ago

hey @NaxAlpha can you try out this fix and let me know if it works

NaxAlpha commented 3 years ago

sure let me try this!

NaxAlpha commented 3 years ago

unfortunately I am getting this error now: image

running the docker image without --gpus all locally gives the same error.

jjmachan commented 3 years ago

this could be because of some nvidia-container-runtime but will need some time to figure this out unfortunately. Which instance_type are you using?

NaxAlpha commented 3 years ago

For this service: ml.g4dn.xlarge

NaxAlpha commented 3 years ago

While I could deploy this service last week on GPU, somehow, it is not working right now. Also, I am not sure if it is related to the current issue, but I will try multipart on another service deployed on a CPU instance. And let you know!

jjmachan commented 3 years ago

Thanks @NaxAlpha!

The API Gateway is still not configured properly for the ImageInput handler. Can you try testing it with the FileInput handler too? Really sorry for these inconveniences but will get these ironed out ASAP.

NaxAlpha commented 3 years ago

Ok just tested a CPU service. The fix is working perfectly!

NaxAlpha commented 3 years ago

BTW CPU service looks something like this: (It was also not working previously because of the same issue)

class Model(bentoml.BentoService):
    @bentoml.api(
        input=MultiFileInput(
            input_names=[
                "image1",
                "image2",
                "config",
            ]
        ),
        batch=False,
    )
    def predict(self, image1: FileLike, image2: FileLike, config: FileLike):
        ...
jjmachan commented 3 years ago

We have figured out a better approach to this problem here #20 hence closing this.