search

zonca / jupyterhub-deploy-kubernetes-jetstream

Configuration files for my tutorials on deploying JupyterHub on top of Kubernetes on XSEDE Jetstream (Openstack)
https://zonca.dev/categories/#jetstream
23 stars 14 forks source link

Autoscaling Kubernetes on Jetstream #13

Closed zonca closed 5 years ago

zonca commented 5 years ago

I have a tutorial on how to scale (i.e. add or remove OpenStack Virtual Machines from the Kubernetes cluster) manually a cluster: https://zonca.github.io/2019/02/scale-kubernetes-jupyterhub-manually.html

Next we would like this process to be automatic, this is well supported on AWS and Google Cloud, but not on OpenStack.

In this issue I'll track general progress on this issue.

Cluster API

It looks like this is the future, but it is progressing quite slowly, see https://github.com/zonca/jupyterhub-deploy-kubernetes-jetstream/issues/14

Cluster Autoscaler

This is the official project that provides autoscaling on AWS and Google Cloud. There used to be no support for OpenStack, they just added it in March 2019, see https://github.com/zonca/jupyterhub-deploy-kubernetes-jetstream/issues/15

zonca commented 5 years ago

@julienchastang @ktyle @rsignell-usgs, here a summary on the current status of autoscaling, I'll do some testing on the new Cluster Autoscaler for Openstack and let you know if it works on Jetstream.

zonca commented 5 years ago

Cluster Autoscaler works pretty well, see https://zonca.github.io/2019/09/kubernetes-jetstream-autoscaler.html We can also do load testing with hubtraf https://zonca.github.io/2019/10/loadtest-jupyterhub.html