Configure most hubs to use CanvasOAuthenticator

ryanlovett commented 1 year ago

We have decided^W generally agreed to switch from using CanvasOAuth on datahub and GenericOAuth on most of the rest to using CanvasOAuth on most hubs. See #3520. , but in short, it generally simplifies authentication.

This would require:

[x] enter the hub URLs in Canvas (bCourses)
[x] document this in https://github.com/berkeley-dsep-infra/datahub/blob/staging/docs/admins/howto/new-hub.rst
[x] update the cookiecutter template to include jupyterhub.hub.config.CanvasOAuthenticator, with client_id, client_secret, and oauth_callback_url.
[x] update the same config in all current genericoauth hubs' secrets/{prod,staging}.
[x] update all current genericoauth hubs' config/common to remove JupyterHub.authenticator_class and GenericOAuthenticator under jupyterhub.hub.config. The former is set to canvas in hub/values.yml.
[ ] remove jupyterhub.hub.services in deployments/datahub/secrets/{prod,staging}.

This should be done after Fall is over and some time before Spring.

balajialg commented 1 year ago

Thanks, @ryanlovett!

yuvipanda commented 1 year ago

This could be something I help oversee before I leave? that would leave the NFS as the primary single point of failure, removing the main datahub as an additional single point of failure for auth during login.

@felder if you can add URLs for all hubs in Canvas as allowed oauth_callback URLs, I can do the rest.

balajialg commented 1 year ago

That's awesome. Thanks @yuvipanda!

yuvipanda commented 1 year ago

Great. @felder if you let me know once you add URLs for all the hubs, I'll do the rest

felder commented 1 year ago

I believe this represents the full list:

$ grep -rinse callback * | grep -v template | grep config
a11y/config/staging.yaml:31:        oauth_callback_url: 'https://a11y-staging.berkeley.edu/hub/oauth_callback'
a11y/config/prod.yaml:25:        oauth_callback_url: 'https://a11y.berkeley.edu/hub/oauth_callback'
cee/config/staging.yaml:28:        oauth_callback_url: 'https://cee-staging.berkeley.edu/hub/oauth_callback'
cee/config/prod.yaml:22:        oauth_callback_url: 'https://cee.berkeley.edu/hub/oauth_callback'
data101/config/staging.yaml:28:        oauth_callback_url: 'https://data101-staging.berkeley.edu/hub/oauth_callback'
data101/config/prod.yaml:22:        oauth_callback_url: 'https://data101.berkeley.edu/hub/oauth_callback'
datahub/config/staging.yaml:11:        oauth_redirect_uri: https://cee-staging.datahub.berkeley.edu/hub/oauth_callback
datahub/config/staging.yaml:15:        oauth_redirect_uri: https://astro-staging.datahub.berkeley.edu/hub/oauth_callback
datahub/config/staging.yaml:19:        oauth_redirect_uri: https://data8-staging.datahub.berkeley.edu/hub/oauth_callback
datahub/config/staging.yaml:23:        oauth_redirect_uri: https://data100-staging.datahub.berkeley.edu/hub/oauth_callback
datahub/config/staging.yaml:27:        oauth_redirect_uri: https://data101-staging.datahub.berkeley.edu/hub/oauth_callback
datahub/config/staging.yaml:31:        oauth_redirect_uri: https://data102-staging.datahub.berkeley.edu/hub/oauth_callback
datahub/config/staging.yaml:35:        oauth_redirect_uri: https://dlab-staging.datahub.berkeley.edu/hub/oauth_callback
datahub/config/staging.yaml:39:        oauth_redirect_uri: https://eecs-staging.datahub.berkeley.edu/hub/oauth_callback
datahub/config/staging.yaml:43:        oauth_redirect_uri: https://ischool-staging.datahub.berkeley.edu/hub/oauth_callback
datahub/config/staging.yaml:47:        oauth_redirect_uri: https://julia-staging.datahub.berkeley.edu/hub/oauth_callback
datahub/config/staging.yaml:51:        oauth_redirect_uri: https://prob140-staging.datahub.berkeley.edu/hub/oauth_callback
datahub/config/staging.yaml:55:        oauth_redirect_uri: https://publichealth-staging.datahub.berkeley.edu/hub/oauth_callback
datahub/config/staging.yaml:59:        oauth_redirect_uri: https://r-staging.datahub.berkeley.edu/hub/oauth_callback
datahub/config/staging.yaml:63:        oauth_redirect_uri: https://stat159-staging.datahub.berkeley.edu/hub/oauth_callback
datahub/config/staging.yaml:67:        oauth_redirect_uri: https://stat20-staging.datahub.berkeley.edu/hub/oauth_callback
datahub/config/staging.yaml:71:        oauth_redirect_uri: https://shiny-staging.datahub.berkeley.edu/hub/oauth_callback
datahub/config/staging.yaml:75:        oauth_redirect_uri: https://a11y-staging.datahub.berkeley.edu/hub/oauth_callback
datahub/config/prod.yaml:17:        oauth_redirect_uri: https://cee.datahub.berkeley.edu/hub/oauth_callback
datahub/config/prod.yaml:21:        oauth_redirect_uri: https://astro.datahub.berkeley.edu/hub/oauth_callback
datahub/config/prod.yaml:25:        oauth_redirect_uri: https://data8.datahub.berkeley.edu/hub/oauth_callback
datahub/config/prod.yaml:29:        oauth_redirect_uri: https://data100.datahub.berkeley.edu/hub/oauth_callback
datahub/config/prod.yaml:33:        oauth_redirect_uri: https://data101.datahub.berkeley.edu/hub/oauth_callback
datahub/config/prod.yaml:37:        oauth_redirect_uri: https://data102.datahub.berkeley.edu/hub/oauth_callback
datahub/config/prod.yaml:41:        oauth_redirect_uri: https://dlab.datahub.berkeley.edu/hub/oauth_callback
datahub/config/prod.yaml:45:        oauth_redirect_uri: https://eecs.datahub.berkeley.edu/hub/oauth_callback
datahub/config/prod.yaml:49:        oauth_redirect_uri: https://ischool.datahub.berkeley.edu/hub/oauth_callback
datahub/config/prod.yaml:53:        oauth_redirect_uri: https://julia.datahub.berkeley.edu/hub/oauth_callback
datahub/config/prod.yaml:57:        oauth_redirect_uri: https://prob140.datahub.berkeley.edu/hub/oauth_callback
datahub/config/prod.yaml:61:        oauth_redirect_uri: https://r.datahub.berkeley.edu/hub/oauth_callback
datahub/config/prod.yaml:65:        oauth_redirect_uri: https://publichealth.datahub.berkeley.edu/hub/oauth_callback
datahub/config/prod.yaml:69:        oauth_redirect_uri: https://stat159.datahub.berkeley.edu/hub/oauth_callback
datahub/config/prod.yaml:73:        oauth_redirect_uri: https://stat20.datahub.berkeley.edu/hub/oauth_callback
datahub/config/prod.yaml:77:        oauth_redirect_uri: https://shiny.datahub.berkeley.edu/hub/oauth_callback
datahub/config/prod.yaml:81:        oauth_redirect_uri: https://a11y.datahub.berkeley.edu/hub/oauth_callback
ischool/config/staging.yaml:31:        oauth_callback_url: 'https://ischool-staging.berkeley.edu/hub/oauth_callback'
ischool/config/prod.yaml:25:        oauth_callback_url: 'https://ischool.berkeley.edu/hub/oauth_callback'

felder commented 1 year ago

This is the sorted list for staging:

$ grep -rinse callback * | grep -v template | grep config | tr -s ' ' | cut -d ' ' -f3 | sed s/\'//g | grep stag | sort
https://a11y-staging.berkeley.edu/hub/oauth_callback
https://a11y-staging.datahub.berkeley.edu/hub/oauth_callback
https://astro-staging.datahub.berkeley.edu/hub/oauth_callback
https://cee-staging.berkeley.edu/hub/oauth_callback
https://cee-staging.datahub.berkeley.edu/hub/oauth_callback
https://data100-staging.datahub.berkeley.edu/hub/oauth_callback
https://data101-staging.berkeley.edu/hub/oauth_callback
https://data101-staging.datahub.berkeley.edu/hub/oauth_callback
https://data102-staging.datahub.berkeley.edu/hub/oauth_callback
https://data8-staging.datahub.berkeley.edu/hub/oauth_callback
https://dlab-staging.datahub.berkeley.edu/hub/oauth_callback
https://eecs-staging.datahub.berkeley.edu/hub/oauth_callback
https://ischool-staging.berkeley.edu/hub/oauth_callback
https://ischool-staging.datahub.berkeley.edu/hub/oauth_callback
https://julia-staging.datahub.berkeley.edu/hub/oauth_callback
https://prob140-staging.datahub.berkeley.edu/hub/oauth_callback
https://publichealth-staging.datahub.berkeley.edu/hub/oauth_callback
https://r-staging.datahub.berkeley.edu/hub/oauth_callback
https://shiny-staging.datahub.berkeley.edu/hub/oauth_callback
https://stat159-staging.datahub.berkeley.edu/hub/oauth_callback
https://stat20-staging.datahub.berkeley.edu/hub/oauth_callback

Also going to throw in https://staging.datahub.berkeley.edu/hub/oauth_callback

Which replaces what is currently defined:

https://cs194-staging.datahub.berkeley.edu/hub/oauth_callback
https://data100-staging.datahub.berkeley.edu/hub/oauth_callback
https://data102-staging.datahub.berkeley.edu/hub/oauth_callback
https://dlab-staging.datahub.berkeley.edu/hub/oauth_callback
https://eecs-staging.datahub.berkeley.edu/hub/oauth_callback
https://julia-staging.datahub.berkeley.edu/hub/oauth_callback
https://prob140-staging.datahub.berkeley.edu/hub/oauth_callback
https://r-staging.datahub.berkeley.edu/hub/oauth_callback
https://staging.datahub.berkeley.edu/hub/oauth_callback
https://stat159-staging.datahub.berkeley.edu/hub/oauth_callback
https://stat89a-staging.datahub.berkeley.edu/hub/oauth_callback
https://w261-staging.datahub.berkeley.edu/hub/oauth_callback

Noting that a11y, cee, data101, and ischool are all defined twice with two different URLs.

@balajialg can you please confirm which ones are correct? I assume it's the ones that end with datahub.berkeley.edu.

felder commented 1 year ago

This is the sorted list for prod:

$ grep -rinse callback * | grep -v template | grep config | tr -s ' ' | cut -d ' ' -f3 | sed s/\'//g | grep -v stag | sort
https://a11y.berkeley.edu/hub/oauth_callback
https://a11y.datahub.berkeley.edu/hub/oauth_callback
https://astro.datahub.berkeley.edu/hub/oauth_callback
https://cee.berkeley.edu/hub/oauth_callback
https://cee.datahub.berkeley.edu/hub/oauth_callback
https://data100.datahub.berkeley.edu/hub/oauth_callback
https://data101.berkeley.edu/hub/oauth_callback
https://data101.datahub.berkeley.edu/hub/oauth_callback
https://data102.datahub.berkeley.edu/hub/oauth_callback
https://data8.datahub.berkeley.edu/hub/oauth_callback
https://dlab.datahub.berkeley.edu/hub/oauth_callback
https://eecs.datahub.berkeley.edu/hub/oauth_callback
https://ischool.berkeley.edu/hub/oauth_callback
https://ischool.datahub.berkeley.edu/hub/oauth_callback
https://julia.datahub.berkeley.edu/hub/oauth_callback
https://prob140.datahub.berkeley.edu/hub/oauth_callback
https://publichealth.datahub.berkeley.edu/hub/oauth_callback
https://r.datahub.berkeley.edu/hub/oauth_callback
https://shiny.datahub.berkeley.edu/hub/oauth_callback
https://stat159.datahub.berkeley.edu/hub/oauth_callback
https://stat20.datahub.berkeley.edu/hub/oauth_callback

Also going to throw in https://datahub.berkeley.edu/hub/oauth_callback

Which replaces what is currently defined:

https://cs194.datahub.berkeley.edu/hub/oauth_callback
https://data100.datahub.berkeley.edu/hub/oauth_callback
https://data102.datahub.berkeley.edu/hub/oauth_callback
https://datahub.berkeley.edu/hub/oauth_callback
https://dlab.datahub.berkeley.edu/hub/oauth_callback
https://eecs.datahub.berkeley.edu/hub/oauth_callback
https://julia.datahub.berkeley.edu/hub/oauth_callback
https://prob140.datahub.berkeley.edu/hub/oauth_callback
https://r.datahub.berkeley.edu/hub/oauth_callback
https://stat159.datahub.berkeley.edu/hub/oauth_callback
https://stat89a.datahub.berkeley.edu/hub/oauth_callback
https://w261.datahub.berkeley.edu/hub/oauth_callback

Noting that a11y, cee, data101, and ischool are all defined twice with two different URLs.

@balajialg can you please confirm which ones are correct? I assume it's the ones that end with datahub.berkeley.edu.

felder commented 1 year ago

@yuvipanda @balajialg I've updated canvas as described above, but would like to remove one of each of the entries that I indicated were defined twice unless they are supposed to be defined twice.

felder commented 1 year ago

@yuvipanda also noting that biology is not in the above lists, adding it as well.

balajialg commented 1 year ago

@felder Thanks for looking into this. Yes - the callback URL which has "datahub.berkeley.edu" is the right one.

yuvipanda commented 1 year ago

Thanks a lot, @felder! I'll try out on some of the smaller hubs next week, and move the major ones after end of semester.

ryanlovett commented 1 year ago

This update requires a newer version of jupyterhub than what is in the image we're using. Per thread in slack, @yuvipanda suggests we use the latest release and anticipates no breaking changes.

yuvipanda commented 1 year ago

Thanks @ryanlovett. I don't think moving to CanvasOAuthenticator from GenericOAuthenticator is blocked on the JupyterHub image bump though - as they currently don't have the groups feature anyway.

ryanlovett commented 1 year ago

@yuvipanda Ideally we should use a JupyterHub image with the fix from https://github.com/jupyterhub/jupyterhub/issues/4017, though we could work around it.

yuvipanda commented 1 year ago

@ryanlovett right! but that is only if we wanna use group functionality right? We can basically extend what we have for datahub to rest of the things.

balajialg commented 1 year ago

@ryanlovett

[x] Bump the version of the hub image
[x] Turn on canvas authenticator

yuvipanda commented 1 year ago

Gonna unassign from myself, as @ryanlovett has agreed to take this all on :)

ryanlovett commented 1 year ago

This is fixed by #4035 and previous commits. I still need to "remove jupyterhub.hub.services in deployments/datahub/secrets/{prod,staging}" (from checklist above), but it is harmless for now. I want to make sure it is easly to fall back on that configuration should something be amiss with the canvas auth.

berkeley-dsep-infra / datahub

Configure most hubs to use CanvasOAuthenticator #3906