search

jupyterhub / zero-to-jupyterhub-k8s

Helm Chart & Documentation for deploying JupyterHub on Kubernetes
https://zero-to-jupyterhub.readthedocs.io
Other
1.56k stars 796 forks source link

Travis -> CircleCI for k8s 1.11 #820

Closed consideRatio closed 6 years ago

consideRatio commented 6 years ago

Using minikube for k8s 1.11 or above requires kubeadm that in turn require systemd, that cannot be installed on Ubuntu 14.04. Therefor minikube requires Ubuntu 16.04 or higher. Travis on the other hand does not support anything newer than Ubuntu 14.04.

Enter CircleCI that supports Ubuntu 18.04, and I have heard @yuvipanda appreciate it in the gitter chat. Is it okay that work towards switching to CircleCI for z2jh?

consideRatio commented 6 years ago

Ah ehm... Okay so CircleCI also use a Ubuntu 14.04 image if executing as "machine", while they can execute prebuilt images in a "docker" mode as compared to "machine" mode. Yikes... Lots to learn...

manics commented 6 years ago

Does CircleCI support external test infrastructure? Thinking in of https://github.com/jupyterhub/zero-to-jupyterhub-k8s/issues/172#issuecomment-404487767

consideRatio commented 6 years ago

@manics I don't know, but I think GitLab does - they speak about how easy it is to work with an external kubernetes cluster. GitLab seem to be praised a lot for its CI/CD. GitLabs CI/CD can be used for GitHub repos as well without forcing a code base migration to GitLab.

manics commented 6 years ago

👍 I'm using Gitlab for our production JupyterHub deployment: https://github.com/IDR/k8s-analysis-deploy. Still some bugs still to be ironed out, but not sure if they're problems with GitLab, Kubernetes or JupyterHub

jgerardsimcock commented 6 years ago

@manics are you able to use gitlab/circle-ci to deploy to your k8s cluster if the build passes? I see you mention it not working? Do you know what the bug is?

manics commented 6 years ago

@jgerardsimcock So far I've only implemented the deployment part, it's not yet a full CI-CD workflow, mostly because I haven't had time to work on some integration tests. I find it's still useful to be able to push an updated configuration to git and have Gitlab automatically trigger a deployment even without good tests.

Most of the bugs seem to be intermittent, e.g. RPC timeouts (presumably a Kubernetes or Gitlab Runner issue, this is a self-provisioned k8s cluster, a standard public cloud offering might work better). I've had loads of problems with the z2jh pre-puller, but I think that's meant to be fixed now, @consideRatio can you confirm?

consideRatio commented 6 years ago

@manics It is a lot more robust now in master than v0.6, but it will become even more robust in #758, but that PR is not merged yet. I need to extract separate PRs for that and a kubespawner PR in order to get things merged, currently it is just too much in one PR to review. Speaking of which, I'd be very happy about reviews on all the upcoming extracted PR's!

In master right now

Prepuller logic is reworked. It now uses a daemonset spreading pods to nodes and waiting for the daemonset status to be "all pods are ready", while in v0.6 it is "lets create a daemonset manually, and then ask all nodes about what images are available right now - until all images are found in the response we should keep waiting...". The latter can fail when the node has more than 50 unique images on it, as the response about what images are available is limited to 50.

In #758 and to be extracted from it I think

A new type of helm hook, allowing for previously found k8s resources of the same name to be deleted, is used to ensure there will at most be one single image-puller being orphaned on helm upgrade failures, and even if this would happen, they will be cleaned up on the next successful helm upgrade.

consideRatio commented 6 years ago

Yuvi suggested we look into this, and now Travis -> CircleCI seems like a change that does not bring anything essential to the table: https://aspenmesh.io/2018/01/building-istio-with-minikube-in-a-container-and-jenkins/