awslabs / multi-model-server

Multi Model Server is a tool for serving neural net models for inference
Apache License 2.0
998 stars 230 forks source link

Added special handling for returning various errors during model initialization #822

Closed vdantu closed 5 years ago

vdantu commented 5 years ago

Before or while filing an issue please feel free to join our slack channel to get in touch with development team, ask questions, find out what's cooking and more!

Issue #, if available:

MMS orchestrators need to be notified of OOM errors which occur during Model Initialization or prediction times. This can be used by the orchestrators to re-distribute the workers on different hosts.

Description of changes:

Testing done:

To run CI tests on your changes refer README.md

By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.