Create a BinderHub helm chart in the infrastructure repo

[sgibson91]

• Structurally, I'd imagine we'd make a 'binderhub' helm-chart, which has a dependency on both binderhub and dask-gateway. We can use condition to disable dask-gateway in future binderhubs that don't need this. A big problem here is the lack of composability in helm, and we will have to duplicate a bunch of things from our basehub and daskhub chart values.yaml files :( Is there any way to avoid this? We'd also need to make sure the z2jh version matches what we have in basehub, and I'm not sure how exactly to do that either.

Originally posted by @yuvipanda in https://github.com/2i2c-org/infrastructure/issues/919#issuecomment-1116347379

• Project Board for this work

Tasks

• [x] Scope and design a binderhub helm chart under the helm-charts folder in this repo

[sgibson91]

I have opened a very basic PR here: https://github.com/2i2c-org/infrastructure/pull/1289

[sgibson91]

Some notes after chatting with @yuvipanda

• We are going to want to reuse some default values for this chart from basehub/daskhub, but helm is not very composable. Potential upstream work in the future may also change how our charts compose against z2jh. Hence we need a short-term hack to deploy this chart without bringing a lot of build complexity that we would have to undo later.
  • [ ] Simplest solution agreed upon was to copy-paste the default values we want, and then write a validation check that runs in CI that fails if the default values for basehub/daskhub have changed without being copied over to binderhub

    • 1402

  • Alternative option is to make this validation a pre-commit hook, but I thought this would be a rabbit hole for me since I haven't written one before and I'll get too excited. CI is the 'boring' option that will get me to move onto the next task more reliably 😅
• [x] We should use https://github.com/sgibson91/bump-helm-deps-action to keep the binderhub chart dependency up-to-date with the upstream tags
  • https://github.com/2i2c-org/infrastructure/pull/1391
• [x] We should improve https://github.com/sgibson91/bump-jhub-image-action to (i) pull from quay.io, and (ii) be compatible with bumping the repo2docker tag in a binderhub chart values
  • https://github.com/2i2c-org/infrastructure/issues/1308
• [x] Test this chart in the 2i2c/pilot-hubs cluster
  • https://github.com/2i2c-org/infrastructure/pull/1404
• RE cryptomining:
  • [x] ask mybinder team about upstreaming the tools they use
  • [x] monero deployment from https://hackmd.io/3_UBXSW3T4Om3wv_buyZ-g
    • https://github.com/2i2c-org/infrastructure/pull/1420
  • [ ] enable cilogon auth for now from https://github.com/2i2c-org/infrastructure/pull/1218

[damianavila]

Potential upstream work in the future may also change how our charts compose against z2jh

Can you drop some thoughts/details about this one? Thanks!

[sgibson91]

Can you drop some thoughts/details about this one? Thanks!

Chris wrote about it here https://github.com/2i2c-org/infrastructure/issues/1382

[damianavila]

Chris wrote about it here https://github.com/2i2c-org/infrastructure/issues/1382

I might be missing something... but I see high-level stuff on that issue (and there in proposal). I was a little bit more interested in the technical details about:

change how our charts compose against z2jh

Feel free to point me out to existing notes/docs about that discussion if those already exist.

[sgibson91]

@damianavila I think the point is that we don't know. I'm not even sure if the BinderHub chart will even continue to exist if you can use repo2docker to built images directly in z2jh 🤷🏻‍♀️ Until that issue I linked to has more technical details, I can't say for sure what the future of this chart will be.

[sgibson91]

Issue I'm having with the binderhub helm chart:

• We want the binderhub helm chart to have the same functionality as basehub.
• To do this we need to reuse the templates from basehub so that we can set up scratch buckets, etc.
• @yuvipanda and I discussed having a symbolic link for the templates from basehub to binderhub.
• This doesn't actually work because:
  • The templates are configured to look for .Values.jupyterhub.custom
  • The binderhub helm chart actually has .Values.binderhub.jupyterhub.custom

How to resolve?

• Instead of symbolic links, actually copy the templates and find/replace all .Values.jupyterhub instances with .Values.binderhub.jupyterhub. Provide a validation check on the template files like we will be doing with the values files.
• Something else?

@yuvipanda would really appreciate your input here

[sgibson91]

@choldgraf and I chatted today and we will open a post on Pangeo's Discourse forum to get input on the urgency of a Binder on GCP in the community. If it's not that urgent, it will probably be better to redirect the operational toil into https://github.com/2i2c-org/infrastructure/issues/1382 instead of developing a million hacks to get a Pangeo Binder up now that will only have to change once that work is underway.

[yuvipanda]

@sgibson91 I think just copying them over is ok for now too.

Alternatively, we just completely ignore basehub and just copy over whatever is in https://github.com/jupyterhub/mybinder.org-deploy, remove what isn't needed and get that going.

[sgibson91]

@yuvipanda and I had a quick chat about moving this forward in a quicker manner. Plan is the following:

• Instead of focusing on what's in our basehub and daskhub config, pull in the config that Pangeo were already using for the Binder last time
• Some further work:
  • [x] Deploy cryptnono (anti-cryptomining) as part of our support chart
    • https://github.com/2i2c-org/infrastructure/pull/1420
  • [x] Terraform update to deploy GCR and push images there (instead of my personal Docker Hub account)
    • https://github.com/2i2c-org/infrastructure/pull/1442
    • https://github.com/2i2c-org/infrastructure/pull/1453
  • [x] Test with examples from pangeo gallery
• Call this "done" until we can move it over to a non-Columbia account with credits and open it up to the Pangeo community
• Move operational toil to the GESIS work

[sgibson91]

• Terraform update to deploy GCR and push images there (instead of my personal Docker Hub account)

@yuvipanda I noticed that our GCP terraform config is already setup to create a GAR, so maybe we should use that instead of GCR?

Transition notes: https://cloud.google.com/artifact-registry/docs/transition/transition-from-gcr

[yuvipanda]

@sgibson91 oooh yes using GAR would be perfect.

[sgibson91]

@yuvipanda I requested your review on https://github.com/2i2c-org/infrastructure/pull/1442

[sgibson91]

I believe https://github.com/2i2c-org/infrastructure/pull/1404 is now ready to be reviewed/merged

[sgibson91]

I also now have config that successfully pushes the built images to GAR instead of my Docker Hub account! But I'd like to get #1404 merged first so I can rebase without force pushing anything.

[sgibson91]

Following PR enables the GAR. It's already deployed though!

• https://github.com/2i2c-org/infrastructure/pull/1453

[sgibson91]

Last task to close this issue:

• https://github.com/2i2c-org/infrastructure/issues/1449

Associated PR:

• https://github.com/2i2c-org/infrastructure/pull/1458

[sgibson91]

Woohoo! I'm calling this one closed! 🎉