[Request deployment] New hub projectpythia.2i2c.cloud

[jmunroe]

We have Cloudbank funding to deploy a Project Pythia/Pangeo JupyterHub and BinderHub on GCP.

Cluster deployment:

• https://github.com/2i2c-org/infrastructure/issues/3652

This is flowing from a grant from NSF:

Rose, Brian E. J., Clyne, John, May, Ryan, Munroe, James, Snyder, Amelia, Eroglu, Orhan, & Tyle, Kevin. (2023). Collaborative Research: GEO OSE TRACK 2: Project Pythia and Pangeo: Building an inclusive geoscience community through accessible, reusable, and reproducible workflows. Zenodo. https://doi.org/10.5281/zenodo.8184298

This JupyterHub will support Project Pythia workflows for use by the broader Pangeo community.

I think we'll want both a persistent storage JupyterHub and ephemeral storage BinderHub both running on the same cluster. This is a pattern of JupyterHub for a smaller set of authorized, core users who create content, and a BinderHub for a much larger user group who do not require persistent storage.

We have a few projects with this access pattern requirement (HPOSS, HHMI, ProjectPythia, Catalyst) so it make some sense to first plan this kind of deployment as being just one instance of a JupyterHub/BinderHub combination on a dedicated cluster.

Phase 3 : Hub setup

There's going to be a number of hubs, and this starts specifying them.

Hub 1: Staging (Ready)

Phase 3.1: Initial setup

Question	Answer	Notes
Name of the hub	staging
Dask gateway?	no
Splash image	https://projectpythia.org/_static/pythia_logo-white-rtext.svg
URL	https://projectpythia.org/
funded_by	NSF	Rose, Brian E. J., Clyne, John, May, Ryan, Munroe, James, Snyder, Amelia, Eroglu, Orhan, & Tyle, Kevin. (2023). Collaborative Research: GEO OSE TRACK 2: Project Pythia and Pangeo: Building an inclusive geoscience community through accessible, reusable, and reproducible workflows. Zenodo. https://doi.org/10.5281/zenodo.8184298
Authentication Mechanism	GitHub (via GitHubOAuthenticator)
Admin Users	@jmunroe @ktyle

At the end of this phase, both 2i2c engineers and the admin users mentioned can login to the hub.

Phase 3.2: Object storage access

Question	Answer	Notes
Scratch bucket enabled?	Yes
Persistent bucket enabled?	no
Requestor pays requests to external buckets allowed?	no

Phase 3.3: Profile List

These are the standard profile list options we want to start with. Can be customized later

Question	Answer	Notes
Pangeo Notebook?	Yes	Use latest available date tag for pangeo-notebook
RStudio (with Rocker)?	No
Allow users to specify any image they want to use?	Yes	Enable unlisted_choice
Max RAM option allowed	30GB	Output of deployer generate resource-allocation choices r5.xlarge --num-allocations 5 should be used
GPU enabled?	yes
Default Interface	JupyterLab
Allow multiple concurrent servers per user?	Yes	Translates to allowNamedServers

Phase 3.4: Authentication tuning (NOT READY)

Question	Answer
Authentication Mechanism	GitHub (via GitHubOAuthenticator)
Org based access?	yes
List of GitHub orgs / teams to be granted access	https://github.com/orgs/ProjectPythia

Phase 3.5: Profile List finetuning

Question	Answer	Notes
Custom image to be specified?	no

Hub 2: prod (Ready)

same as staging but for production

Hub 3: binder-like hub (Not Ready)

https://binder.projectpythia.2i2c.cloud

Equivalent functionality to https://binder.projectpythia.org

Authentication: requires a discussion with engineering on options.

[ktyle]

@jmunroe @yuvipanda @AIDEA775 Is the Jupyterhub instance ready? If so, could you please grant my GitHub ID access to the hub?

[jmunroe]

Hi @ktyle -- as soon as the hub is deployed, we will let you know. This hub is @AIDEA775 next task to pick up.

[AIDEA775]

Hi @ktyle and @jmunroe! I changed the logo to the blue version (https://projectpythia.org/_static/images/logos/pythia_logo-blue-rtext.svg) because the white version is not visible with the white background.

The staging and prod hubs are deployed! prod: https://projectpythia.2i2c.cloud/ staging: https://staging.projectpythia.2i2c.cloud/

But before you try to log in, I will temporarily need admin permission in the GitHub organization to do the last step:

https://infrastructure.2i2c.org/hub-deployment-guide/configure-auth/github-orgs/#granting-access-to-the-oauth-app

[ktyle]

Hi @AIDEA775 sounds good ... regarding the admin permission, is this something you need our organization (ProjectPythia) to provide, or are you waiting for someone in your 2i2c org to do this?

[yuvipanda]

@ktyle yes, you'd have to grant that to him!

So close to complete :)

[ktyle]

ok @AIDEA775 give it a try and let me know! Thanks!

[AIDEA775]

Thank you @ktyle! Everything is ready. Can you log in?

[ktyle]

@AIDEA775 nice!! Yes, I can log into both instances! Can any member of the ProjectPythia organization now log into these hubs, or do we need to provide a list of users for you?

[ktyle]

Looks like folks can log in so I think we're good! Thanks again!

[jmunroe]

Thanks @AIDEA775 !!