search

2i2c-org / infrastructure

Infrastructure for configuring and deploying our community JupyterHubs.
https://infrastructure.2i2c.org
BSD 3-Clause "New" or "Revised" License
105 stars 64 forks source link

[New Hub] University of Washington ICESAT Hackweeks #962

Closed choldgraf closed 2 years ago

choldgraf commented 2 years ago

Hub Description

This is a hub for a team at the University of Washington for several upcoming ICESAT Hackweeks.

The environment should be pretty standard for a research workflow - the most complex bits here are the GitHub Teams authentication, and deploying this in an AWS cluster.

Community Representative(s)

Important dates

Target start date

ASAP (so that they can git it a whirl and play around with the setup)

Preferred Cloud Provider

Amazon Web Services

Preferred Location of the Cloud Resources

us-west-2

Do you have your own billing account?

Hub Authentication Type

Other (may not be possible, please specify in comments)

Hub logo

No response

Hub logo URL

No response

Hub image service

FIND OUT. They wish to auto-deploy a Docker image, but not sure if they have one yet.

Hub image

FIND OUT. They wish to auto-deploy a Docker image, but not sure if they have one yet.

Extra features you'd like to enable

Other relevant information

Authentication: They'd like GitHub Teams authentication within the following GitHub organization: https://github.com/ICESAT-2HackWeek

Per-user resources

Hub ID

No response

Hub Cluster

No response

Hub URL

No response

Hub Type

Research Hub (though scalable dask cluster isn't needed)

Tasks to deploy the hub

choldgraf commented 2 years ago

Hey @scottyhq - would you mind clarifying some of the empty boxes above so that we can get this hub deployed? Also a few specific questions:

scottyhq commented 2 years ago

Thanks @choldgraf! So excited to collaborate with 2i2c for https://icesat-2.hackweek.io. Suggested amendments and notes below:

Community Representative(s)

@scottyhq, @JessicaS11, @aaarendt

Important dates

First Hackweek: March 21-25, 2022

Do you have your own billing account?

No. But we have AWS credits from NASA + AWS Sustainability Data Initiative. I'll send an email with the credit code.

Hub Authentication Type

We'd like to limit access to this GitHub Organization Team https://github.com/orgs/ICESAT-2HackWeek/teams/jupyterhub-2022

Hub logo URL

Not sure if we can use multiple, if not the first one would be best: https://icesat-2hackweek.github.io/assets/images/ICESat2.png https://escience.washington.edu/wp-content/uploads/2015/10/eScience_Logo_HR.png

Hub image service

auto-deploy

Hub image

We're pushing a Docker image to these two repositories (just need to autodeploy 'latest' from either one) https://hub.docker.com/r/uwhackweek/icesat2 https://quay.io/repository/uwhackweek/icesat2

consideRatio commented 2 years ago

For the credit voucher granting AWS credits, see https://2i2c.freshdesk.com/a/tickets/73.

yuvipanda commented 2 years ago

I was tempted to run this as a https://github.com/yuvipanda/jupyterhub-multicluster-kubespawner but I think it's too early still, and we should just run a dedicated hub instead.

choldgraf commented 2 years ago

No worries - so I think that means that our next steps are:

?

yuvipanda commented 2 years ago

@choldgraf yeah I've updated the task body with what I think of as the TODO.

yuvipanda commented 2 years ago

@choldgraf can you give me billing rights on the 2i2c sandbox AWS project (id 746653422107)? I've created an AWS organization with it as the management account, and I'll need to disable credit sharing before I can redeem the AWS voucher provided to us. Alternatively, you can go to https://console.aws.amazon.com/billing/home?region=us-east-1#/ logged in as your choldgraf@2i2c.org sandbox user, and disable credit sharing under preferences?

yuvipanda commented 2 years ago

https://docs.aws.amazon.com/IAM/latest/UserGuide/tutorial_billing.html?icmpid=docs_iam_console#tutorial-billing-step2 has instructions on providing billing access.

yuvipanda commented 2 years ago

@scottyhq I've set it up at https://uwhackweeks.2i2c.cloud/, including the GitHub authentication with teams. Can you try it out and let me know if it works?

If you'd prefer a different domain name, let me know as well.

scottyhq commented 2 years ago

Amazing! Thank you @yuvipanda. I just tried logging in but get 403: Forbidden, Looks like you have NOT been added to the list of allowed users for this hub. Please contact the hub administrators. I just sent you an invite to the github organization team in case it helps for troubleshooting.

yuvipanda commented 2 years ago

@scottyhq i think it needs permissions to read teams. I've just requested that - an email should've come to the owners of the org.

yuvipanda commented 2 years ago

@scottyhq does it work for you now?

scottyhq commented 2 years ago

@scottyhq does it work for you now?

I'm in! Much appreciated @yuvipanda. As we kick the tires, should we follow up with any issues here, or should further conversation happen elsewhere?

choldgraf commented 2 years ago

@scottyhq - usually we try to get the hub into a working state per the needs of the community using it. This often requires a few back-and-forths, and once it seems good enough, we close the "new hub" issue and start spot-checking other changes via support@2i2c.org. Want to try that and see how it goes?

Also just a note that @yuvipanda is about to go on vacation, so it might be better to ping support@2i2c.org so others have visibility as well.

scottyhq commented 2 years ago

@choldgraf @yuvipanda it seems the authentication still isn't working as expected. Only I am able to login, and not other members of our github org team. Perhaps I was only able to get in because I'm listed as a hub 'admin'? If the team-based authentication doesn't work, we can just fall back to the entire org.

sgibson91 commented 2 years ago

Hi @scottyhq I've found a bug in your helm config - I will push a fix

sgibson91 commented 2 years ago

For posterity: the allowed_users key had been added to the helm chart config. Under normal authentication circumstances, not providing that key would allow anyone on GitHub to login to the hub and obviously we don't want to allow that for abuse prevention reasons. However, using the allowed_users key in tandem with GitHub Org/Team authentication means that no users are granted access to the hub unless they have been added via the admin panel, even if they are valid members of the org/team used for authentication. We remove this key, the problem should resolve itself.

I am less sure on the mixture of GitHub Org and GitHub Teams auth currently used in your helm chart, specifically the mixing of the read: user and read:org scopes. (I believe the thinking behind this is to allow anyone in the 2i2c org access, but only those in your specific team access.) I've left it how it is for now, but if you see anything weird, let me know and I'll scope it to just Teams instead of this mix.

scottyhq commented 2 years ago

Thanks @sgibson91 ! Confirmed this is working now for other users.

I am less sure on the mixture of GitHub Org and GitHub Teams auth currently used in your helm chart, specifically the mixing of the read: user and read:org scopes.

Yeah, I've always been confused by the combination of what is required minimal scopes for the authenticator, and whether org members must set their profile to 'public' or not. Feel free to fiddle with it if you want, our plan for the next week is to operate in a testing mode and then add more people for tutorial development.

sgibson91 commented 2 years ago

whether org members must set their profile to 'public' or not.

Using the read: org scope, they definitely don't. But it does mean we get more info about your org in the handshake than just that team, and that could be a security issue in some scenarios.

scottyhq commented 2 years ago

But it does mean we get more info about your org in the handshake than just that team, and that could be a security issue in some scenarios.

Gotcha, if you want to disable read: org temporarily I can test whether it is in fact required to read the team membership for our organization.

One other thought after quickly testing things out - is it easy to remove the 'RStudio' logo and radio button from the hub landing page?

We don't plan on doing anything with R during our event and the option might confuse users as it is not part of our docker image. Unless there is a separate 2i2c-managed Rstudio default image that can be pointed at? Currently selecting this launch option leads to a 404: Not found. You are requesting a page that does not exist!. Again, not a huge issue, but figured I'd flag it.

sgibson91 commented 2 years ago

@GeorgianaElena could we have a branch of the pilots-homepage repo without the RStudio logo for the above request? 👆🏻

GeorgianaElena commented 2 years ago

@sgibson91, yes, should be possible! I'm looking into it.

GeorgianaElena commented 2 years ago

Update: RStudio logo + button have been removed for both the staging and prod hubs :rocket:

GeorgianaElena commented 2 years ago

Hi @scottyhq! Just wanted to double-check if you wanted a dedicated scalable dask Cluster with dask gateway for this hub. It is my understanding from the the top comment, that you don't.

However, the hub was configured to support this and I wasn't sure if it was intended. A missing dask_gateway in the user image causes some test failures with the current setup I believe.

scottyhq commented 2 years ago

Thanks for checking @GeorgianaElena, dask_gateway wasn't part of the plan for this hub, so I think we're good to go! Feel free to close this issue and I'll follow up with support@2i2c.org if anything else comes up!

scottyhq commented 2 years ago

Actually,,, one more thing :) I just tried to update the Docker image to quay.io/uwhackweek/icesat2:2022.02.15 by going to https://uwhackweeks.2i2c.cloud/services/configurator/. The 'Submit' button seems unresponsive and no confirmation is given, but it does actually register the new image!

Screen Shot 2022-02-18 at 1 41 25 PM
consideRatio commented 2 years ago

@scottyhq I note that this seems reported in https://github.com/yuvipanda/jupyterhub-configurator/issues/5#issue-850285786 - I'll make a +1 comment about the value of having that submit button feedback. I've wished for it as well.

choldgraf commented 2 years ago

Hey all - I'm gonna close this one since I believe the hub is in a steady state now, but we can keep track of follow-up items in subsequent issues for events etc.