search

jupyterhub / zero-to-jupyterhub-k8s

Helm Chart & Documentation for deploying JupyterHub on Kubernetes
https://zero-to-jupyterhub.readthedocs.io
Other
1.55k stars 794 forks source link

Document deploying JupyterHub to Jetstream2 using Kubernetes and ClusterAPI #3354

Open jmunroe opened 8 months ago

jmunroe commented 8 months ago

The most recent work on deploying JupyterHub to Jetstream2 using Kubernetes is covered by @zonca 's excellent tutorials and blog posts summarized in https://www.zonca.dev/posts/2023-09-21-gateways-2023-dask-jupyterhub

In recent discussions with the Jetstream2 team, the next iteration of a 'best practice' for deploying JuptyerHub will likely be based around ClusterAPI (https://cluster-api.sigs.k8s.io)

This issue marks a common location for iterating and discussing the steps required to support ClusterAPI on Jetstream2.

When that iterative process had progress, these recommendations for using Kubernetes should be added to the list of cloud providers documented in https://z2jh.jupyter.org/en/stable/kubernetes/setup-kubernetes.html

cboettig commented 5 months ago

@julianpistorius any updates on jupyterhub-based access to jetstream2?

julianpistorius commented 4 months ago

Hi @cboettig! Apologies for the delay. I have been traveling and got behind on my GitHub alerts. Yes, I have been evaluating options. I will meet with Jeremy & the team next week about my findings, and will update this issue then.

cboettig commented 4 months ago

@julianpistorius Hooray! sounds great, keep us posted!

cboettig commented 4 months ago

@julianpistorius any updates?

cboettig commented 4 months ago

also maybe an aside, but is their any connection between Jetstream2 GPU and the https://nairrpilot.org/ efforts? Or is the latter exclusively focused on the commercial cloud?

julianpistorius commented 4 months ago

Hi @cboettig!

I've been trying to debug an issue with Cluster-API which is preventing us from getting to the next step. I think I'm close, but it's hard to say with so many moving parts. I have a few things to try and will keep you updated.

As for NAIRR, yes Jetstream2 is involved: https://news.iu.edu/live/news/35829-ius-cloud-computing-resource-receives-49m-to

You will see Indiana Jetstream2 GPU in the resources: https://submit-nairr.xras.org/resources

And you can request it when you submit a proposal: https://submit-nairr.xras.org/opportunities/533375/requests/new

Are you interested in submitting a proposal?

cboettig commented 4 months ago

thanks @julianpistorius , great! Keep us posted.

Re NAIRR, excellent! Yes, actually just put in an education call proposal to try this in my classroom, though it looks like at the moment it only lists SDSC's Prototype National Research Platform (PNRP) Classroom (which I listed) or Vocareum AI as possible providers on that side. It is great to see Jetstream2 being listed on the research side (along with many other academic cloud platforms)

julianpistorius commented 4 months ago

@cboettig Definitely off-topic now, but I've been told that we're working on making Jetstream2 available for NAIRR education proposals.

julianpistorius commented 2 months ago

Update: We figured out the gotchas, and have deployed a few Kubernetes clusters on Jetstream2 using Cluster API. We also deployed a basic JupyterHub. I feel comfortable enough with this process to write documentation now.

(O/T @cboettig Jetstream2 is now available as a classroom resource for NAIRR)

cboettig commented 2 months ago

@julianpistorius amazing! I'm really excited to try this out both for our research program and teaching with NAIRR