substratusai / lingo

Lightweight ML model proxy and autoscaler for kubernetes
https://www.substratus.ai
Apache License 2.0
96 stars 6 forks source link

Only get stats and autoscale deployments with model annotation #77

Open samos123 opened 5 months ago

samos123 commented 5 months ago

Currently this will be seen in the logs

Kubernetes service which has no deployment:

2024/02/10 19:51:42 Average for deployment: kubernetes: 0 (ceil: 0), current wait count: 0
2024/02/10 19:51:42 SetDesiredScale(0), current: -1, min: 0, max: 0
2024/02/10 19:51:42 Scaling up to 0 immediately                                                                 
2024/02/10 19:51:42 Scaling model "kubernetes": 0                                                               
2024/02/10 19:51:42 error scaling: get: Deployment.apps "kubernetes" not found
2024/02/10 19:51:42 Is leader, autoscaling       
2024/02/10 19:51:42 Average for deployment: lingo: 0

Lingo which has a deployment but no annotation:

2024/02/10 19:51:45 Average for deployment: lingo: 0 (ceil: 0), current wait count: 0
2024/02/10 19:51:45 SetDesiredScale(0), current: -1, min: 0, max: 0
2024/02/10 19:51:45 Scaling up to 0 immediately
2024/02/10 19:51:45 Scaling model "lingo": 0
2024/02/10 19:51:45 error scaling: not scaling deployment "lingo": deployment has no models annotation

Expected behavior:

Note that solving #59 might solve this issue as well. Both could potentially be tackled in the same PR, but doesn't have to be. There might a much smaller simpler fix for this issue vs #59