awslabs / multi-model-server

Multi Model Server is a tool for serving neural net models for inference
Apache License 2.0
984 stars 230 forks source link

How to handle invalid input! #1022

Open xuweidongkobe opened 11 months ago

xuweidongkobe commented 11 months ago

Given a MMS custom service that expects JSON as input, If the input is invalid JSON such as { "image": "I missed a quote} MMS will return { "code": 500, "type": "InternalServerException", "message": "Worker died." } and reload model. Is something can do to verify the accuracy of input to avoid such direct returns and model reload when it cannot be modified input?

xuweidongkobe commented 11 months ago

@maaquib