jupyterhub / oauthenticator

OAuth + JupyterHub Authenticator = OAuthenticator
https://oauthenticator.readthedocs.io
BSD 3-Clause "New" or "Revised" License
413 stars 365 forks source link

[Generic] GET request for userdata_url built incorrectly #654

Open lyh16 opened 1 year ago

lyh16 commented 1 year ago

Bug description

oauthenticator keeps adding a colon(":") to the end of the specified userdata_url during OAuth authentication, causing a 403 error and ultimately leading to an Error 500 on the-littlest-jupyterhub.

Expected behaviour

OAuth authentication should be done successfully.

GET https://<REDACTED>/application/o/userinfo/

Actual behaviour

the-littlest-jupyterhub fails to login via GenericOAuthenticator.

(Emphasis on colon at the end)
GET https://<REDACTED>/application/o/userinfo/:

How to reproduce

  1. Populate config.yaml accordingly.
  2. Add a few customizations to jupyterhub_config.d.
  3. Run tljh-config reload

Your personal set up

Full environment ``` # paste output of `pip freeze` or `conda list` here alembic==1.11.1 async-generator==1.10 attrs==23.1.0 certifi==2019.11.28 certipy==0.1.3 cffi==1.15.1 chardet==3.0.4 cryptography==41.0.2 dbus-python==1.2.16 distro==1.4.0 distro-info===0.23ubuntu1 greenlet==2.0.2 idna==2.8 importlib-metadata==6.8.0 importlib-resources==6.0.0 Jinja2==3.1.2 jsonschema==4.18.4 jsonschema-specifications==2023.7.1 jupyter-telemetry==0.1.0 jupyterhub==4.0.1 Mako==1.2.4 MarkupSafe==2.1.3 oauthenticator==16.0.3 oauthlib==3.2.2 packaging==23.1 pamela==1.1.0 pkgutil-resolve-name==1.3.10 prometheus-client==0.17.1 pycparser==2.21 PyGObject==3.36.0 pyOpenSSL==23.2.0 python-apt==2.0.1+ubuntu0.20.4.1 python-dateutil==2.8.2 python-json-logger==2.0.7 referencing==0.30.0 requests==2.22.0 requests-unixsocket==0.2.0 rpds-py==0.9.2 ruamel.yaml==0.17.32 ruamel.yaml.clib==0.2.7 six==1.14.0 SQLAlchemy==2.0.19 ssh-import-id==5.10 tornado==6.3.2 traitlets==5.9.0 typing-extensions==4.7.1 unattended-upgrades==0.1 urllib3==1.25.8 zipp==3.16.2 ```
Configuration ```yaml # config.yaml users: admin: - user_environment: default_app: jupyterlab services: cull: timeout: 2630000 auth: type: oauthenticator.generic.GenericOAuthenticator ``` ```python # jupyterhub_config.d/admin_access.py c.JupyterHub.admin_access = True # jupyterhub_config.d/authentik.py c.GenericOAuthenticator.client_id = "" c.GenericOAuthenticator.client_secret = "" c.GenericOAuthenticator.oauth_callback_url = "https:///hub/oauth_callback" c.GenericOAuthenticator.authorize_url = "https:///application/o/authorize/" c.GenericOAuthenticator.token_url = "https:///application/o/token/" c.GenericOAuthenticator.userdata_url = "https:///application/o/userinfo/" c.GenericOAuthenticator.logout_redirect_url = "/application/o//end-session/" c.LocalAuthenticator.create_system_users = True c.GenericOAuthenticator.login_service = "Authentik" c.GenericOAuthenticator.username_key = "username" c.GenericOAuthenticator.allowed_groups = {"", ""} c.GenericOAuthenticator.admin_users = {""} ```
Logs ``` # paste relevant logs here, if any Jul 20 05:50:47 hostname systemd[1]: Started jupyterhub.service. Jul 20 05:50:54 hostname python3[7758]: [I 2023-07-20 05:50:54.023 JupyterHub app:2859] Running JupyterHub version 4.0.1 Jul 20 05:50:54 hostname python3[7758]: [I 2023-07-20 05:50:54.024 JupyterHub app:2889] Using Authenticator: oauthenticator.generic.GenericOAuthenticator-16.0.3 Jul 20 05:50:54 hostname python3[7758]: [I 2023-07-20 05:50:54.024 JupyterHub app:2889] Using Spawner: tljh.user_creating_spawner.UserCreatingSpawner Jul 20 05:50:54 hostname python3[7758]: [I 2023-07-20 05:50:54.024 JupyterHub app:2889] Using Proxy: jupyterhub_traefik_proxy.fileprovider.TraefikFileProviderProxy-1.1.0 Jul 20 05:50:54 hostname python3[7758]: [I 2023-07-20 05:50:54.037 JupyterHub app:1664] Loading cookie_secret from /opt/tljh/state/jupyterhub_cookie_secret Jul 20 05:50:54 hostname python3[7758]: [I 2023-07-20 05:50:54.356 JupyterHub app:1984] Not using allowed_users. Any authenticated user will be allowed. Jul 20 05:50:54 hostname python3[7758]: [I 2023-07-20 05:50:54.512 JupyterHub fileprovider:99] Creating the dynamic configuration file: /opt/tljh/state/rules/rules.toml Jul 20 05:50:54 hostname python3[7758]: [I 2023-07-20 05:50:54.580 JupyterHub app:2928] Initialized 0 spawners in 0.068 seconds Jul 20 05:50:54 hostname python3[7758]: [I 2023-07-20 05:50:54.592 JupyterHub metrics:278] Found 0 active users in the last ActiveUserPeriods.twenty_four_hours Jul 20 05:50:54 hostname python3[7758]: [I 2023-07-20 05:50:54.596 JupyterHub metrics:278] Found 0 active users in the last ActiveUserPeriods.seven_days Jul 20 05:50:54 hostname python3[7758]: [I 2023-07-20 05:50:54.598 JupyterHub metrics:278] Found 0 active users in the last ActiveUserPeriods.thirty_days Jul 20 05:50:54 hostname python3[7758]: [I 2023-07-20 05:50:54.599 JupyterHub app:3142] Not starting proxy Jul 20 05:50:54 hostname python3[7758]: [I 2023-07-20 05:50:54.604 JupyterHub app:3178] Hub API listening on http://127.0.0.1:15001/hub/ Jul 20 05:50:54 hostname python3[7758]: [I 2023-07-20 05:50:54.604 JupyterHub app:3189] Starting managed service cull-idle Jul 20 05:50:54 hostname python3[7758]: [I 2023-07-20 05:50:54.604 JupyterHub service:385] Starting service 'cull-idle': ['/opt/tljh/hub/bin/python3', '-m', 'jupyterhub_idle_culler', '--timeout=2630000', '--cull-every=60', '--concurrency=5', '--max-age=0'] Jul 20 05:50:54 hostname python3[7758]: [I 2023-07-20 05:50:54.605 JupyterHub service:133] Spawning /opt/tljh/hub/bin/python3 -m jupyterhub_idle_culler --timeout=2630000 --cull-every=60 --concurrency=5 --max-age=0 Jul 20 05:50:54 hostname python3[7758]: [I 2023-07-20 05:50:54.611 JupyterHub app:3247] JupyterHub is now running, internal Hub API at http://127.0.0.1:15001/hub/ Jul 20 05:50:54 hostname python3[7758]: [I 2023-07-20 05:50:54.854 JupyterHub log:191] 200 GET /hub/api/ (cull-idle@127.0.0.1) 34.77ms Jul 20 05:50:54 hostname python3[7758]: [I 2023-07-20 05:50:54.891 JupyterHub log:191] 200 GET /hub/api/users?state=[secret] (cull-idle@127.0.0.1) 35.09ms Jul 20 05:50:55 hostname python3[7758]: [I 2023-07-20 05:50:55.277 JupyterHub log:191] 200 GET /hub/api (@127.0.0.1) 1.21ms Jul 20 05:51:11 hostname python3[7758]: [I 2023-07-20 05:51:11.770 JupyterHub log:191] 302 GET /hub/ -> /hub/login?next=%2Fhub%2F (@172.24.0.1) 0.84ms Jul 20 05:51:12 hostname python3[7758]: [I 2023-07-20 05:51:12.066 JupyterHub log:191] 200 GET /hub/login?next=%2Fhub%2F (@172.24.0.1) 68.60ms Jul 20 05:51:13 hostname python3[7758]: [I 2023-07-20 05:51:13.935 JupyterHub oauth2:102] OAuth redirect: https:///hub/oauth_callback Jul 20 05:51:13 hostname python3[7758]: [I 2023-07-20 05:51:13.936 JupyterHub log:191] 302 GET /hub/oauth_login?next=%2Fhub%2F -> https:///application/o/authorize/?response_type=code&redirect_uri=https%3A%2F%2Faica.imtl.re.kr%2Fhub%2Foauth_callback&client_id=KR1LQrhMOKA6StFviUxt1iB3zjIsrUznTxScSpwZ&state=[secret] (@172.24.0.1) 1.70ms Jul 20 05:51:16 hostname python3[7758]: [E 2023-07-20 05:51:16.351 JupyterHub oauth2:620] Error Fetching user info... 403 GET https:///application/o/userinfo/: Jul 20 05:51:16 hostname python3[7758]: [E 2023-07-20 05:51:16.351 JupyterHub web:1871] Uncaught exception GET /hub/oauth_callback?code=1693c6b2b52c41b98a5b8c8323193d3c&state=eyJzdGF0ZV9pZCI6ICI0MjFlODQyMWFjMWM0OWMxYTNjYzJlZGU3YTkzMmYyNyIsICJuZXh0X3VybCI6ICIvaHViLyJ9 (172.24.0.1) Jul 20 05:51:16 hostname python3[7758]: HTTPServerRequest(protocol='http', host='', method='GET', uri='/hub/oauth_callback?code=1693c6b2b52c41b98a5b8c8323193d3c&state=eyJzdGF0ZV9pZCI6ICI0MjFlODQyMWFjMWM0OWMxYTNjYzJlZGU3YTkzMmYyNyIsICJuZXh0X3VybCI6ICIvaHViLyJ9', version='HTTP/1.1', remote_ip='172.24.0.1') Jul 20 05:51:16 hostname python3[7758]: Traceback (most recent call last): Jul 20 05:51:16 hostname python3[7758]: File "/opt/tljh/hub/lib/python3.8/site-packages/tornado/web.py", line 1786, in _execute Jul 20 05:51:16 hostname python3[7758]: result = await result Jul 20 05:51:16 hostname python3[7758]: File "/opt/tljh/hub/lib/python3.8/site-packages/oauthenticator/oauth2.py", line 202, in get Jul 20 05:51:16 hostname python3[7758]: user = await self.login_user() Jul 20 05:51:16 hostname python3[7758]: File "/opt/tljh/hub/lib/python3.8/site-packages/jupyterhub/handlers/base.py", line 826, in login_user Jul 20 05:51:16 hostname python3[7758]: authenticated = await self.authenticate(data) Jul 20 05:51:16 hostname python3[7758]: File "/opt/tljh/hub/lib/python3.8/site-packages/jupyterhub/auth.py", line 492, in get_authenticated_user Jul 20 05:51:16 hostname python3[7758]: authenticated = await maybe_future(self.authenticate(handler, data)) Jul 20 05:51:16 hostname python3[7758]: File "/opt/tljh/hub/lib/python3.8/site-packages/oauthenticator/oauth2.py", line 958, in authenticate Jul 20 05:51:16 hostname python3[7758]: user_info = await self.token_to_user(token_info) Jul 20 05:51:16 hostname python3[7758]: File "/opt/tljh/hub/lib/python3.8/site-packages/oauthenticator/oauth2.py", line 873, in token_to_user Jul 20 05:51:16 hostname python3[7758]: return await self.httpfetch( Jul 20 05:51:16 hostname python3[7758]: File "/opt/tljh/hub/lib/python3.8/site-packages/oauthenticator/oauth2.py", line 655, in httpfetch Jul 20 05:51:16 hostname python3[7758]: return await self.fetch( Jul 20 05:51:16 hostname python3[7758]: File "/opt/tljh/hub/lib/python3.8/site-packages/oauthenticator/oauth2.py", line 621, in fetch Jul 20 05:51:16 hostname python3[7758]: raise e Jul 20 05:51:16 hostname python3[7758]: File "/opt/tljh/hub/lib/python3.8/site-packages/oauthenticator/oauth2.py", line 600, in fetch Jul 20 05:51:16 hostname python3[7758]: resp = await self.http_client.fetch(req, **kwargs) Jul 20 05:51:16 hostname python3[7758]: tornado.httpclient.HTTPClientError: HTTP 403: Forbidden Jul 20 05:51:16 hostname python3[7758]: Jul 20 05:51:16 hostname python3[7758]: [E 2023-07-20 05:51:16.442 JupyterHub log:183] { Jul 20 05:51:16 hostname python3[7758]: "Host": "", Jul 20 05:51:16 hostname python3[7758]: "User-Agent": "Mozilla/5.0 (X11; Linux x86_64; rv:102.0) Gecko/20100101 Firefox/102.0", Jul 20 05:51:16 hostname python3[7758]: "Accept": "text/html,application/xhtml+xml,application/xml;q=0.9,image/avif,image/webp,*/*;q=0.8", Jul 20 05:51:16 hostname python3[7758]: "Accept-Encoding": "gzip", Jul 20 05:51:16 hostname python3[7758]: "Accept-Language": "en-US,en;q=0.5", Jul 20 05:51:16 hostname python3[7758]: "Cdn-Loop": "cloudflare", Jul 20 05:51:16 hostname python3[7758]: "Cf-Connecting-Ip": "", Jul 20 05:51:16 hostname python3[7758]: "Cf-Ipcountry": "", Jul 20 05:51:16 hostname python3[7758]: "Cf-Ray": "7e98ee68ac2219c9-KIX", Jul 20 05:51:16 hostname python3[7758]: "Cf-Visitor": "{\"scheme\":\"https\"}", Jul 20 05:51:16 hostname python3[7758]: "Cf-Warp-Tag-Id": "2f5ca5a4-957d-4be2-95c5-fc080a93c01d", Jul 20 05:51:16 hostname python3[7758]: "Cookie": "_xsrf=[secret]; oauthenticator-state=[secret]", Jul 20 05:51:16 hostname python3[7758]: "Dnt": "1", Jul 20 05:51:16 hostname python3[7758]: "Priority": "u=1", Jul 20 05:51:16 hostname python3[7758]: "Sec-Fetch-Dest": "document", Jul 20 05:51:16 hostname python3[7758]: "Sec-Fetch-Mode": "navigate", Jul 20 05:51:16 hostname python3[7758]: "Sec-Fetch-Site": "same-site", Jul 20 05:51:16 hostname python3[7758]: "Sec-Gpc": "1", Jul 20 05:51:16 hostname python3[7758]: "Upgrade-Insecure-Requests": "1", Jul 20 05:51:16 hostname python3[7758]: "X-Forwarded-For": "", Jul 20 05:51:16 hostname python3[7758]: "X-Forwarded-Host": "", Jul 20 05:51:16 hostname python3[7758]: "X-Forwarded-Port": "80", Jul 20 05:51:16 hostname python3[7758]: "X-Forwarded-Proto": "http", Jul 20 05:51:16 hostname python3[7758]: "X-Forwarded-Server": "hostname", Jul 20 05:51:16 hostname python3[7758]: "X-Real-Ip": "" Jul 20 05:51:16 hostname python3[7758]: } Jul 20 05:51:16 hostname python3[7758]: [E 2023-07-20 05:51:16.442 JupyterHub log:191] 500 GET /hub/oauth_callback?code=[secret]&state=[secret] (@172.24.0.1) 1103.29ms ```
welcome[bot] commented 1 year ago

Thank you for opening your first issue in this project! Engagement like this is essential for open source projects! :hugs:
If you haven't done so already, check out Jupyter's Code of Conduct. Also, please try to follow the issue template as it helps other other community members to contribute more effectively. welcome You can meet the other Jovyans by joining our Discourse forum. There is also an intro thread there where you can stop by and say Hi! :wave:
Welcome to the Jupyter community! :tada:

mangecoeur commented 1 year ago

I'm am also having problems with userdata fetching but I checked the code and it is not adding a colon, if you look at the code in oatuh2.py:620

app_log.error(f"Error {label} {e.code} {req.method} {url}: {message}")

the colon is part of the error message. The problem is there is no message added which might better explain the error.

consideRatio commented 1 year ago

Thanks @mangecoeur for figuring that out! Okay, so there is an issue with userdata fetching, but unrelated to the colon added as part of an error message.

I think at this point I lack information to debug the actual issue then.

mangecoeur commented 1 year ago

P.S I at least found out why I am not getting an error message: it seems most Oauth providers return errors as Json with some information, but in my case connecting to Microsoft ADFS based server it returns an HTTP error 401 with no body. I was able to extract the access token and test it out in an HTTP client which gives access to the WWW-Authenticate header that contains more info (error MSIS9921 in my case)

mangecoeur commented 1 year ago

P.P.S not sure this is the best place to store this knowledge but with the help of https://serverfault.com/questions/1109792/setting-up-oidc-with-adfs-invalid-userinfo-request I figured out that you need to define resource=urn:microsoft:userinfo for the inital authorize request. I finally found the (somewhat hidden) extra_authorize_params setting so now I have c.GenericOAuthenticator.extra_authorize_params = {"resource": "urn:microsoft:userinfo"} and it works!