Kubernetes Autoscaling

[osterman]

what

• How do we add resource scaling triggers based on custom metrics?

See #20 (asked by @Nuru)

[osterman]

@Nuru can you help me understand the kinds of autoscaling you have in mind?

For example:

• Scale based on number of items in a queue
• Scale based on requests-per-second
• Scale based on time-of-day

Kubernetes provides the ability to autoscale certain kinds of resources (e.g. the number Pods in a ReplicaSet) as well as the ability to scale out the number of Nodes in the cluster. Kubernetes supports both, however, generally scaling out Nodes is a slower operation, while scaling out Pods can be nearly instantaneous. Out-of-the-box, kubernetes can scale on the standard kinds of metrics (CPU, Memory), but it's possible to do custom instrumentation.

TL;DR

Autoscaling Pods is the easiest way. Basically, a small app is written up and deployed as a controller in the cluster. This can be as simple as a bash script or as complicated as a go app.

Essentially, they automate the the following command (which can also be run manually):

 kubectl scale -n $namespace --replicas=$desiredPods deployment/$deployment

Read more

• Horizontal Pod Autoscaler
• Scale Based on RabbitMQ Queue Size (bash)
• Scale Based on AMQP Queue Size (go)

[Nuru]

@osterman The above is a great start on the documentation I am asking for.

I am mainly asking specifically how we would connect some metric we generate to our specific Kubernetes control plane to scale pods and nodes up and down in a way that coordinates with other scaling signals (rather than overrides them).

For example, let's say that we decide we are going to have a pod that holds long-lived TCP connections to users and we want to limit the number of open sockets in the pod to 10,000 and the number of open sockets in the node to 50,000. Assume we have a shell script sockcount that returns the number of open sockets as reported by the kernel it is running on.

Questions I have include:

• Where (URL, file, etc.) do I set/check the apiserver flags documented at Configure the Aggregation Layer
• Where Horizontal Pod Autoscaler says <path-to-kubeconfig> and <ip-address-of-apiserver> where do we find the values to substitute in?
• Once we "write a small app" what do we need to do to "deploy it as a controller in the cluster"?
• How do we coordinate this scaler with others, so that when this scaler says it only needs 1 node, we do not take the whole system down to only 1 node?