search

substratusai / lingo

Lightweight ML model proxy and autoscaler for kubernetes
https://www.substratus.ai
Apache License 2.0
102 stars 6 forks source link

Handle model backend deployment deletions #42

Open alpe opened 7 months ago

alpe commented 7 months ago

When a deployment for a model backend is deleted from the k8s cluster all pending requests should be aborted and new requests rejected with 404.

nstogner commented 7 months ago

Should we mark this as resolved after merging #44?

alpe commented 7 months ago

There is also the case where a model was removed from an annotation. This is not handled, yet