Technical draft (state of the art) about the BinderHub/JupyterHub project

damianavila commented 2 years ago

Context

The goal we have with the notes is to provide an overview of the UX and tech available via tljh-repo2docker / binderhub, and how they couple to various core jupyterhub functionalities and specific implementations of Spawners for example.

The value we see for such an overview is that we think it would be helpful when trying to scope what we are to do going onwards more clearly. We hope that such an overview can for example help us realize better what can be re-used to meet goals etc.

Proposal

[x] Write the draft
[x] Share it with the team to receive feedback

Updates and actions

[x] Explore tljh-r2d codebase

damianavila commented 2 years ago

I think it would be valuable to also include in the discussion (if possible and if it does not make the scope bigger than it should be for a first iteration) any remarkable ideas from the existing experience in https://github.com/gesiscss/persistent_binderhub (as another opinionated data point at the time to deliver the experience/s we are looking for).

arnim commented 2 years ago

https://discourse.jupyter.org/t/a-persistent-binderhub-deployment/2865

arnim commented 2 years ago

With the same caveats as above

if possible and if it does not make the scope bigger than it should be for a first iteration

and if there would be interest, I think it could also be meaningful to ask tljh-repo2docker admins/users about their feedback :)

consideRatio commented 2 years ago

Hi all, here is an update!

I've made https://hackmd.io/@29EjZnuAQx-ME7Olgi2JXw/r1BcEzda5 that includes an overview of things going on in binderhub and tljh-repo2docker, along with a draft idea on a standalone piece of what could possibly be developed.

I hope to discuss this with @yuvipanda @sgibson91 @damianavila in a meeting later today.

arnim commented 2 years ago

Thank you for the overview document. :)

Is there an example on how we could import (and potentially launch) via a link (e.g. http://baseurl.org/v2/gh/binder-examples/requirements/master) some repository? The use-case I'm thinking about could be some kind of gally that uses mybinder for short try-outs and the new JHub import functionality if there is a need to work longer periods on the code.

sgibson91 commented 2 years ago

The use-case I'm thinking about could be some kind of gally that uses mybinder

Why not use mybinder.org then for the short try-outs, and then import the env into your repo2docker-enabled JupyterHub when you've decided you want to use it? Why do these two pieces need to be within the same JupyterHub?

arnim commented 2 years ago

@sgibson91 this would be the idea :) I was just wondering what the equivalent import link for the new r2d JHub importer would be.

sgibson91 commented 2 years ago

Ah ok, I misunderstood your question. I don't think we have scoped that piece yet.

sgibson91 commented 2 years ago

Live update from the meeting: The current proposal is a building block, and not the whole solution. There are pieces missing from this that need to build on top of it that needs further information, e.g., from the UI/UX research. (This probably explains why we do not have an answer for the URL yet 😉 )

damianavila commented 2 years ago

Some notes from the meeting:

There is a need for a more in-depth exploration of the different existing options.
Given the simplicity of the tljh-r2d codebase, @consideRatio is going to explore that project a little bit more to see what are the valuable pieces that we might take to build the "composable building block" [CBB from now ] @sgibson91 was mentioned above.
This is still exploratory work to fully understand the different options and how to build the CCB we are looking for (essentially a repo2docker JHub service).
We think the CBB is the right scope for a first iteration.
The PBHub experience should be built on top of this CBB if we want a long-term sustainable story.
We think building on top of the current BinderHub chart is risky given the current maintainability issues on this project and this is why we want to push forward from a simpler and more sustainable point of origin (CBB).

arnim commented 2 years ago

Thank you for the update :+1:

I think ensuring maintainability and a long-term sustainable story is key. This IMO extends to the necessary support/endorsement from the wider community. You are in the best position to assess which CBBs from the ecosystem we can reuse and which we may need to add. I think our funding is flexible enough also to improve CBBs that are shared with the traditional BHub & JHub if needed.

The important things from my side are that the resulting system is simple to use; i.e. usable by everyone that knows how to launch a binder and can create a Keycloak (or similar) account, as well as easy to deploy for admins. Furthermore, existing galleries for binder resources should be able to point to this new type of enhanced JHub for imports. Finally, in the development of the pBHub we benefited greatly from the circulating ideas and discussions around (re-) introducing binder features into JHub, and I would love if we could similarly incorporate experiences from the pBHub.

damianavila commented 2 years ago

I think our funding is flexible enough also to improve CBBs that are shared with the traditional BHub & JHub if needed.

Super, thanks for confirming that!

The important things from my side are that the resulting system is simple to use; i.e. usable by everyone that knows how to launch a binder and can create a Keycloak (or similar) account, as well as easy to deploy for admins. Furthermore, existing galleries for binder resources should be able to point to this new type of enhanced JHub for imports. Finally, in the development of the pBHub we benefited greatly from the circulating ideas and discussions around (re-) introducing binder features into JHub, and I would love if we could similarly incorporate https://github.com/gesiscss/persistent_binderhub/pull/24#issuecomment-1119512478 from the pBHub.

Thanks for listing out all these items, @arnim! This is exactly the sort of information we want to gather from the UI/UX research we are pushing on #1597. In fact, I think (and I have raised this before in internal conversations) that you and your people should be one of the subjects taking the interview process we are going to build. I guess you are open to being a participant in those? Can I count you in? 😉

arnim commented 2 years ago

We had some User Stories for the pBHub. Maybe they can be of help for #1597.

damianavila commented 1 year ago

The technical draft was provided in https://github.com/2i2c-org/infrastructure/issues/1598#issuecomment-1209281405.

2i2c-org / infrastructure