search

zonca / jupyterhub-deploy-kubernetes-jetstream

Configuration files for my tutorials on deploying JupyterHub on top of Kubernetes on XSEDE Jetstream (Openstack)
https://zonca.dev/categories/#jetstream
23 stars 14 forks source link

Autoscaling Kubernetes on Jetstream with the Cluster API #14

Open zonca opened 5 years ago

zonca commented 5 years ago

Address https://github.com/zonca/jupyterhub-deploy-kubernetes-jetstream/issues/13 using https://github.com/kubernetes-sigs/cluster-api

zonca commented 5 years ago

My comment from December 2018:

it looks interesting, it would allow to declare and manage Openstack resources with declararive K8S style APIs. This would replace kubespray completely I think.

You run minikube locally and then it connects to Openstack to create and manage resources. This is the bootstrapping machine.

Autoscale is supported as well, K8S talks back to the bootstrapping machine to request more Openstack resources. So that better run in a separate VM on jetstream.

Otherwise can be moved into master. The advantage of a separate VM is that it could be used to fix something that breakes on master.

It is in early development stage, reconsider again in a few months.

zonca commented 5 years ago

Checked their documentation again, now the deployment strategy is a lot clearer, but still essential features like networking and floating-ip setup are being implemented, see for example https://github.com/kubernetes-sigs/cluster-api-provider-openstack/issues/205

It looks like we can install minikube with the none driver which installs a 1-node Kubernetes cluster directly on a Jetstream VM, from that we bootstrap the production Kubernetes cluster and then we keep that VM around so that it can increase or decrease the number of OpenStack virtual machines based on load.

It is still too early to switch to this, too many features we get from kubespray are still missing. Re-evaluate later on.

zonca commented 4 years ago

currently Cluster Autoscaler with Magnum works fine. Will reconsider this in the future if useful, closing for now.

zonca commented 4 years ago

it looks like there is some support for openstack now in the Cluster API:

https://cluster-api.sigs.k8s.io/user/quick-start.html

See the Openstack provider: https://github.com/kubernetes-sigs/cluster-api-provider-openstack

It seems like it supports Train, most probably also Rocky:

image

@jlf599 Is Jetstream at IU still on Rocky? are there plans to update anytime soon?

jlf599 commented 4 years ago

We’re on Stein. Both TACC and IU, I believe. Probably will go to Train in June or July.

Jeremy Fischer Manager, Jetstream Cloud Jetstream (NSF Award #ACI-1445604) UITS Research Technologies jeremy@iu.edumailto:jeremy@iu.edu

On Apr 6, 2020, at 5:12 PM, Andrea Zonca notifications@github.com<mailto:notifications@github.com> wrote:

it looks like there is some support for openstack now in the Cluster API:

https://cluster-api.sigs.k8s.io/user/quick-start.html

See the Openstack provider: https://github.com/kubernetes-sigs/cluster-api-provider-openstack

It seems like it supports Stein, most probably also Rocky:

[image]https://user-images.githubusercontent.com/383090/78605550-9dc58980-7810-11ea-93fa-a0e67b835bf7.png

@jlf599https://github.com/jlf599 Is Jetstream at IU still on Rocky? are there plans to update anytime soon?

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHubhttps://github.com/zonca/jupyterhub-deploy-kubernetes-jetstream/issues/14#issuecomment-610039413, or unsubscribehttps://github.com/notifications/unsubscribe-auth/ACZWV7ZI7R6GWUVU25VH7ELRLJAUTANCNFSM4HHKBQ6Q.