search

aws / sagemaker-inference-toolkit

Serve machine learning models within a 🐳 Docker container using 🧠 Amazon SageMaker.
Apache License 2.0
372 stars 82 forks source link

Local development WorkerLifeCycle skip #41

Open danielmapar opened 4 years ago

danielmapar commented 4 years ago

Is your feature request related to a problem? Please describe. Whenever I test my API locally I have to wait for about 5 min for the WorkerLifeCycle to finish. That slowdowns development immensely.

Describe the solution you'd like I would like a config of sorts to initialize the server faster (maybe a dummy worker) so I can test my inference function faster without waiting for 5 min

Describe alternatives you've considered Right now I made a separate Python script to simulate the call to my model (with the payload), but it is far from ideal

nadiaya commented 4 years ago

You can overwrite MMS configuration through environment variables: https://github.com/awslabs/multi-model-server/blob/master/docs/configuration.md#environment-variables

Though I am not sure whether it's possible for WorkerLifeCycle and going to delegate this question to MMS team.