search

yuvipanda / jupyterhub-multicluster-kubespawner

Experimental kubespawner to spawn user pods into many different kubernetes clusters
BSD 3-Clause "New" or "Revised" License
12 stars 1 forks source link

Single Server spawned with multicluster giving 404 Not Found #1

Open stubclan opened 2 years ago

stubclan commented 2 years ago

Hey @yuvipanda first of all really thanks for writing this multicluster which is great use case for us and really works for our use case.

We are using our own image for single server and it runs fine on JupyterHub itself but while deploying to a separate cluster using multicluster kubespawner it just gives 404 for the UI. Was wondering if you could give any insight on this?

image

I tried just running jupyter lab there and it runs fine but running jupyterhub-singleuser --ip=0.0.0.0 --port=8888 --SingleUserNotebookApp.default_url=/lab just makes it give 404 error like above

yuvipanda commented 2 years ago

Thanks for opening this and trying the project out! Can you provide me logs from both the hub pod as well as the user pod? That would help!

stubclan commented 2 years ago

@yuvipanda I am trying to make it similar to singlespawner in terms of env variables and few overrides. But getting 404 constantly both from hub and when doing port forward on singleserver

multikubespawner server pod logs - 

[D 2022-05-24 13:36:44.937 SingleUserNotebookApp handlers:836] Path components/jquery-ui/themes/smoothness/jquery-ui.min.css served from /opt/conda/lib/python3.9/site-packages/notebook/static/components/jquery-ui/themes/smoothness/jquery-ui.min.css
[D 2022-05-24 13:36:44.937 SingleUserNotebookApp handlers:836] Path components/jquery-typeahead/dist/jquery.typeahead.min.css served from /opt/conda/lib/python3.9/site-packages/notebook/static/components/jquery-typeahead/dist/jquery.typeahead.min.css
[D 2022-05-24 13:36:44.938 SingleUserNotebookApp handlers:836] Path style/style.min.css served from /opt/conda/lib/python3.9/site-packages/notebook/static/style/style.min.css
[D 2022-05-24 13:36:44.939 SingleUserNotebookApp handlers:836] Path components/es6-promise/promise.min.js served from /opt/conda/lib/python3.9/site-packages/notebook/static/components/es6-promise/promise.min.js
[D 2022-05-24 13:36:44.939 SingleUserNotebookApp handlers:836] Path components/react/react.production.min.js served from /opt/conda/lib/python3.9/site-packages/notebook/static/components/react/react.production.min.js
[D 2022-05-24 13:36:44.939 SingleUserNotebookApp handlers:836] Path components/react/react-dom.production.min.js served from /opt/conda/lib/python3.9/site-packages/notebook/static/components/react/react-dom.production.min.js
[D 2022-05-24 13:36:44.940 SingleUserNotebookApp handlers:836] Path components/create-react-class/index.js served from /opt/conda/lib/python3.9/site-packages/notebook/static/components/create-react-class/index.js
[D 2022-05-24 13:36:44.940 SingleUserNotebookApp handlers:836] Path components/requirejs/require.js served from /opt/conda/lib/python3.9/site-packages/notebook/static/components/requirejs/require.js
[W 2022-05-24 13:36:44.941 SingleUserNotebookApp log:189] 404 GET / (@127.0.0.1) 28.35ms
[D 2022-05-24 13:36:45.015 SingleUserNotebookApp auth:505] No user identified
[D 2022-05-24 13:36:45.017 SingleUserNotebookApp log:189] 200 GET /user/danilo_ferreira1/static/components/jquery-ui/themes/smoothness/jquery-ui.min.css?v=fb45616eef2c454960f91fcd2a04efeda84cfacccf0c5d741ba2793dc1dbd6d3ab01aaae6485222945774c7d7a9a2e9fb87e0d8ef1ea96893aa6906147a371bb (@127.0.0.1) 2.70ms
[D 2022-05-24 13:36:45.127 SingleUserNotebookApp auth:505] No user identified
[D 2022-05-24 13:36:45.128 SingleUserNotebookApp log:189] 200 GET /user/danilo_ferreira1/static/style/style.min.css?v=b092d0a2da5df36f2b073ddb4eafcd6c8094c4fa21b6dcd3f7185ce16c04ad66424083d785b81607a26b4b85b69a560574ada7db75262c886655f99d651c482e (@127.0.0.1) 2.12ms
[D 2022-05-24 13:36:45.129 SingleUserNotebookApp auth:505] No user identified
[D 2022-05-24 13:36:45.130 SingleUserNotebookApp auth:505] No user identified
[D 2022-05-24 13:36:45.134 SingleUserNotebookApp handlers:836] Path custom.css served from /opt/conda/lib/python3.9/site-packages/notebook/static/custom/custom.css
[D 2022-05-24 13:36:45.135 SingleUserNotebookApp log:189] 200 GET /user/danilo_ferreira1/custom/custom.css (@127.0.0.1) 5.97ms
[D 2022-05-24 13:36:45.136 SingleUserNotebookApp log:189] 200 GET /user/danilo_ferreira1/static/components/jquery-typeahead/dist/jquery.typeahead.min.css?v=5edf53bf6bb9c3b1ddafd8594825a7e2ed621f19423e569c985162742f63911c09eba2c529f8fb47aebf27fafdfe287d563347f58c1126b278189a18871b6a9a (@127.0.0.1) 5.97ms
[D 2022-05-24 13:36:45.137 SingleUserNotebookApp auth:505] No user identified
[D 2022-05-24 13:36:45.137 SingleUserNotebookApp auth:505] No user identified

hub logs - 

[D 2022-05-24 13:40:19.706 JupyterHub base:328] Refreshing auth for danilo_ferreira1
[D 2022-05-24 13:40:19.711 JupyterHub users:783] Not updating activity for <User(danilo_ferreira1 1/1 running)>: 2022-05-24T13:35:31.916202Z < 2022-05-24T13:35:33.143000Z
[D 2022-05-24 13:40:19.711 JupyterHub users:804] Not updating server activity on danilo_ferreira1/: 2022-05-24T13:35:31.916202Z < 2022-05-24T13:35:33.143000Z
[I 2022-05-24 13:40:19.713 JupyterHub log:189] 200 POST /hub/api/users/danilo_ferreira1/activity (danilo_ferreira1@10.171.175.253) 10.71ms
[D 2022-05-24 13:40:20.946 JupyterHub log:189] 200 GET /hub/health (@10.171.175.140) 1.60ms
[D 2022-05-24 13:40:20.946 JupyterHub log:189] 200 GET /hub/health (@10.171.175.140) 1.47ms
pod/jupyter-sujan-aryal condition met
[D 2022-05-24 13:40:22.945 JupyterHub log:189] 200 GET /hub/health (@10.171.175.140) 1.20ms
[D 2022-05-24 13:40:24.944 JupyterHub log:189] 200 GET /hub/health (@10.171.175.140) 0.72ms
[D 2022-05-24 13:40:26.944 JupyterHub log:189] 200 GET /hub/health (@10.171.175.140) 0.89ms
[D 2022-05-24 13:40:28.944 JupyterHub log:189] 200 GET /hub/health (@10.171.175.140) 0.83ms
[D 2022-05-24 13:40:30.945 JupyterHub log:189] 200 GET /hub/health (@10.171.175.140) 1.35ms
[D 2022-05-24 13:40:30.946 JupyterHub log:189] 200 GET /hub/health (@10.171.175.140) 1.29ms
[D 2022-05-24 13:40:32.944 JupyterHub log:189] 200 GET /hub/health (@10.171.175.140) 0.83ms
pod/jupyter-danilo-ferreira1 condition met

All the env variables are provided and there's no particular error message. On the single server pod, it does say No user identified so I assumed it was due to some env or token but couldn't really find the answer. If you have any clue please do help.

Also we are adding somethings to multicluster so it works similar to single, can create a PR if you think its beneficial.

Thank you