search

jupyterhub / team-compass

A repository for team interaction, syncing, and handling meeting notes across the JupyterHub ecosystem.
http://jupyterhub-team-compass.readthedocs.io
62 stars 33 forks source link

Who is interested in collaborating with NFDI around spawning user sessions across multiple clusters? #644

Open choldgraf opened 1 year ago

choldgraf commented 1 year ago

In a recent collaboration cafe we had a presentation from a team at NFDI that is building out a JupyterHub service across much of Germany. @arnim and several others were represented here (Arnim can you cc others as you wish?).

They are interested in enhancing JupyterHub's ability to spawn user sessions across many clusters and/or hubs, so that a single hub could be the launching point for multiple other hubs, potentially in many other clusters.

In the meeting, we agreed that some pieces of this functionality could be beneficial to build into JupyterHub or Z2JH, as many communities want the ability to spawn user sessions across different kinds of infrastructure. @minrk noted that we could try to align on a canonical implementation that JupyterHub recommends, developed in partnership with the NFDI team, and they could then build on top of it for their service. I think @consideRatio and @mriduls were also interested.

Question: who is interested in this?

References

yuvipanda commented 1 year ago

https://github.com/yuvipanda/jupyterhub-multicluster-kubespawner is my take on this. I'd love for us to find funding to finish this.

Ah sorry i see you have already linked it :D

arnim commented 1 year ago

Hey everyone, thanks for your help & thoughts on this! I think one of the central points here is that integrating a national project, like the Jupyter-NFDI, in an international context is something we should plan from the very beginning. We are currently on a by-weekly schedule for drafting the proposal and I'm sure that we will put something nice together :)

Let's stay in contact on this!

cc @berndflemisch, @bhagemeier

kreuzert commented 1 year ago

Hey, Unfortunately, I couldn't join the last collaboration cafe. I am one of the administrators of the mentioned JupyterHub service. I have extracted our Spawner class to an extra repo ( https://github.com/kreuzert/jupyterhub-backendspawner ). With the next update of our JupyterHub service, we will use this one to start JupyterLabs on different systems.

@yuvipanda that's a cool Spawner. But we also want to start JupyterLabs on HPC Systems (and possibly other things, too). That's why we need a more generic approach.

yuvipanda commented 2 months ago

@kreuzert I just took a quick look at outpost and I think I love this model! I am going to try to find some time to dive into it some more, and if it works out I'll probably archive my repo and point people to outpost.

kreuzert commented 2 months ago

@yuvipanda That sounds great. Just let me know, if you or anyone else wants to have a short introduction. Feedback and contributions are always welcome.