As a project, I want my processing cluster to scale up and down depending on processing needs

[mike-gangl]

Scaling to enable:

• on-demand SPS work
• Work based scaling (x jobs are waiting, scale up)
• idle based scale down
• Scaling limits (Limits to how much something can scale up or down based deploy-time configs)

[LucaCinquini]

Drew to investigate options for auto-scaling in Kubernetes:

o Pod Autoscaler: https://kubernetes.io/docs/tasks/run-application/horizontal-pod-autoscale/ o Cluster Auto Scaler: https://kubernetes.io/docs/tasks/run-application/horizontal-pod-autoscale/ o Any way to auto-scale a DaemonSet?

Does any of the above supports custom metrics for auto-scaling? (like the number of jobs in a RabbitMQ queue?)