search

jupyterhub / nbgitpuller

Jupyter server extension to sync a git repository one-way to a local path
https://nbgitpuller.readthedocs.io
BSD 3-Clause "New" or "Revised" License
205 stars 84 forks source link

Change in query string forwarding behavior from 1.2.X to 4.1.X #350

Closed aolney closed 1 month ago

aolney commented 2 months ago

Bug description

In an older version of JupyterHub (TLJH circa spring 2020; JupyterLab 1.2.14), I could distribute an nbgitpuller link like this:

https://saturn.olney.ai/hub/user-redirect/git-pull?repo=https%3A%2F%2Fgithub.com%2Fmemphis-iis%2Fdatawhys-intern-notebooks-2023&urlpath=lab%2Ftree%2Fdatawhys-intern-notebooks-2023%2FDescriptive-statistics.ipynb%3Flog%3Dhttps%3A%2F%2Flogging2.olney.ai%2Fdatawhys%2Flog%26bl%3Dpy&branch=master

and it would resolve to this:

https://saturn.olney.ai/user/aolney/lab?log=https%3A%2F%2Flogging2.olney.ai%2Fdatawhys%2Flog&bl=py

This was very useful because it allowed parameters to be passed in to configure extensions. For example, I have a Blockly extension that opens a Blockly editor in split pane view when it sees bl=py.

However, in a newer version of JupyterHub (TLJH circa spring 2024; JupyterLab 4.1.6), the same nbgitpuller link is resolved to:

https://saturn.olney.ai/user/aolney/lab/tree/datawhys-intern-notebooks-2023

How to reproduce

As above

Expected behaviour

As above

Actual behaviour

As above

Your personal set up

TLJH https://tljh.jupyter.org/en/latest/install/custom-server.html

OS old:

Distributor ID: Ubuntu
Description:    Ubuntu 18.04.6 LTS
Release:        18.04
Codename:       bionic

channels:
  - defaults
dependencies:
  - conda=4.8.1
  - python=3.7
  - jupyterlab=1.2.14
  - nodejs
  - ptvsd
  - xeus-python
  - jupyterlab-git
  - ipympl
  - plotly==4.9
  - jupyterhub-ldapauthenticator
  - jupyterhub-singleuser==1.1.0
  - r-base[version='>=4']
  - r-irkernel
  - r-stringi
  - mamba
  - ca-certificates
  - certifi
  - openssl
  - r-essentials
  - r-car
  - r-emmeans
prefix: /opt/tljh/user

OS new:

Distributor ID: Ubuntu
Description:    Ubuntu 20.04.4 LTS
Release:        20.04
Codename:       focal

channels:
  - conda-forge
dependencies:
  - python=3.10
  - mamba==1.4.1
  - conda==23.1.0
  - pip
  - setuptools=65
  - numpy
  - pandas
  - plotly
  - scikit-learn
  - r-irkernel
  - ipywidgets
  - jupyterlab-git
  - nbdime
  - markdown
  - statsmodels
  - pygraphviz
  - python-graphviz
  - r-modeest
  - r-corrplot
  - r-parsnip
  - r-rlang
  - r-vctrs
  - r-glue
  - r-tidyverse
  - r-tibble
  - geopandas
  - r-tidymodels
  - r-caret
  - r-kknn
  - r-corrr
  - r-janitor
  - r-ggally
  - r-factoextra
  - r-ggfortify
  - r-psych
  - r-glmnet
  - r-rpart.plot
  - r-partykit
  - r-baguette
  - r-ranger
  - r-vip
  - r-remotes
  - r-xgboost
  - r-kernlab
  - r-klar
  - r-naivebayes
  - r-stacks
  - r-furrr
  - r-mlbench
  - r-scales
  - affine
  - sphinx
  - shapely
  - python-kaleido
  - seaborn
  - r-neuralnet
  - conda-build
  - ca-certificates
  - certifi
  - openssl
prefix: /opt/tljh/user
Configuration **Old** ```python # jupyterhub_config.py ``` **New** ```python # jupyterhub_config.py c = get_config() #noqa ```

Diffing the logs shows the redirect behavior is identical (as logged)

Logs **OLD** ``` Apr 23 14:30:32 braintrust python3[1783]: [I 2024-04-23 14:30:32.258 JupyterHub log:174] 302 GET /hub/user-redirect/git-pull?repo=https%3A%2F%2Fgithub.com%2Fmemphis-iis%2Fdatawhys-intern-notebooks-2023&urlpath=lab%2Ftree%2Fdatawhys-intern-notebooks-2023%2FDescriptive-statistics.ipynb%3Flog%3Dhttps%3A%2F%2Flogging2.olney.ai%2Fdatawhys%2Flog%26bl%3Dpy&branch=master -> /hub/spawn/aolney?next=%2Fuser%2Faolney%2Fgit-pull%3Frepo%3Dhttps%253A%252F%252Fgithub.com%252Fmemphis-iis%252Fdatawhys-intern-notebooks-2023%26urlpath%3Dlab%252Ftree%252Fdatawhys-intern-notebooks-2023%252FDescriptive-statistics.ipynb%253Flog%253Dhttps%253A%252F%252Flogging2.olney.ai%252Fdatawhys%252Flog%2526bl%253Dpy%26branch%3Dmaster (aolney@10.101.244.254) 11.01ms Apr 23 14:30:32 braintrust python3[1783]: [W 2024-04-23 14:30:32.270 JupyterHub base:637] Redirecting /hub/spawn/aolney?next=%2Fuser%2Faolney%2Fgit-pull%3Frepo%3Dhttps%253A%252F%252Fgithub.com%252Fmemphis-iis%252Fdatawhys-intern-notebooks-2023%26urlpath%3Dlab%252Ftree%252Fdatawhys-intern-notebooks-2023%252FDescriptive-statistics.ipynb%253Flog%253Dhttps%253A%252F%252Flogging2.olney.ai%252Fdatawhys%252Flog%2526bl%253Dpy%26branch%3Dmaster to /hub/user/aolney/git-pull?repo=https%3A%2F%2Fgithub.com%2Fmemphis-iis%2Fdatawhys-intern-notebooks-2023&urlpath=lab%2Ftree%2Fdatawhys-intern-notebooks-2023%2FDescriptive-statistics.ipynb%3Flog%3Dhttps%3A%2F%2Flogging2.olney.ai%2Fdatawhys%2Flog%26bl%3Dpy&branch=master. For sharing public links, use /user-redirect/ Apr 23 14:30:32 braintrust python3[1783]: [I 2024-04-23 14:30:32.270 JupyterHub pages:145] User is running: aolney Apr 23 14:30:32 braintrust python3[1783]: [I 2024-04-23 14:30:32.271 JupyterHub log:174] 302 GET /hub/spawn/aolney?next=%2Fuser%2Faolney%2Fgit-pull%3Frepo%3Dhttps%253A%252F%252Fgithub.com%252Fmemphis-iis%252Fdatawhys-intern-notebooks-2023%26urlpath%3Dlab%252Ftree%252Fdatawhys-intern-notebooks-2023%252FDescriptive-statistics.ipynb%253Flog%253Dhttps%253A%252F%252Flogging2.olney.ai%252Fdatawhys%252Flog%2526bl%253Dpy%26branch%3Dmaster -> /hub/user/aolney/git-pull?repo=https%3A%2F%2Fgithub.com%2Fmemphis-iis%2Fdatawhys-intern-notebooks-2023&urlpath=lab%2Ftree%2Fdatawhys-intern-notebooks-2023%2FDescriptive-statistics.ipynb%3Flog%3Dhttps%3A%2F%2Flogging2.olney.ai%2Fdatawhys%2Flog%26bl%3Dpy&branch=master (aolney@10.101.244.254) 4.59ms Apr 23 14:30:32 braintrust python3[1783]: [I 2024-04-23 14:30:32.284 JupyterHub log:174] 302 GET /hub/user/aolney/git-pull?repo=https%3A%2F%2Fgithub.com%2Fmemphis-iis%2Fdatawhys-intern-notebooks-2023&urlpath=lab%2Ftree%2Fdatawhys-intern-notebooks-2023%2FDescriptive-statistics.ipynb%3Flog%3Dhttps%3A%2F%2Flogging2.olney.ai%2Fdatawhys%2Flog%26bl%3Dpy&branch=master -> /user/aolney/git-pull?repo=https%3A%2F%2Fgithub.com%2Fmemphis-iis%2Fdatawhys-intern-notebooks-2023&urlpath=lab%2Ftree%2Fdatawhys-intern-notebooks-2023%2FDescriptive-statistics.ipynb%3Flog%3Dhttps%3A%2F%2Flogging2.olney.ai%2Fdatawhys%2Flog%26bl%3Dpy&branch=master&redirects=1 (aolney@10.101.244.254) 4.02ms Apr 23 14:30:39 braintrust python3[1783]: [I 2024-04-23 14:30:39.303 JupyterHub log:174] 200 POST /hub/api/users/aolney/activity (aolney@127.0.0.1) 15.23ms ``` **NEW** ``` Apr 23 19:27:05 datawhys python3[3169912]: [I 2024-04-23 19:27:05.585 JupyterHub log:192] 302 GET /hub/user-redirect/git-pull?repo=https%3A%2F%2Fgithub.com%2Fmemphis-iis%2Fdatawhys-intern-notebooks-2023&urlpath=lab%2Ftree%2Fdatawhys-intern-notebooks-2023%2FDescriptive-statistics.ipynb%3Flog%3Dhttps%3A%2F%2Flogging2.olney.ai%2Fdatawhys%2Flog%26bl%3Dpy&branch=master -> /hub/spawn/aolney?next=%2Fuser%2Faolney%2Fgit-pull%3Frepo%3Dhttps%253A%252F%252Fgithub.com%252Fmemphis-iis%252Fdatawhys-intern-notebooks-2023%26urlpath%3Dlab%252Ftree%252Fdatawhys-intern-notebooks-2023%252FDescriptive-statistics.ipynb%253Flog%253Dhttps%253A%252F%252Flogging2.olney.ai%252Fdatawhys%252Flog%2526bl%253Dpy%26branch%3Dmaster (aolney@10.101.244.254) 4.49ms Apr 23 19:27:05 datawhys python3[3169912]: [W 2024-04-23 19:27:05.595 JupyterHub base:801] Redirecting /hub/spawn/aolney?next=%2Fuser%2Faolney%2Fgit-pull%3Frepo%3Dhttps%253A%252F%252Fgithub.com%252Fmemphis-iis%252Fdatawhys-intern-notebooks-2023%26urlpath%3Dlab%252Ftree%252Fdatawhys-intern-notebooks-2023%252FDescriptive-statistics.ipynb%253Flog%253Dhttps%253A%252F%252Flogging2.olney.ai%252Fdatawhys%252Flog%2526bl%253Dpy%26branch%3Dmaster to /hub/user/aolney/git-pull?repo=https%3A%2F%2Fgithub.com%2Fmemphis-iis%2Fdatawhys-intern-notebooks-2023&urlpath=lab%2Ftree%2Fdatawhys-intern-notebooks-2023%2FDescriptive-statistics.ipynb%3Flog%3Dhttps%3A%2F%2Flogging2.olney.ai%2Fdatawhys%2Flog%26bl%3Dpy&branch=master. For sharing public links, use /user-redirect/ Apr 23 19:27:05 datawhys python3[3169912]: [I 2024-04-23 19:27:05.595 JupyterHub pages:153] User is running: aolney Apr 23 19:27:05 datawhys python3[3169912]: [I 2024-04-23 19:27:05.595 JupyterHub log:192] 302 GET /hub/spawn/aolney?next=%2Fuser%2Faolney%2Fgit-pull%3Frepo%3Dhttps%253A%252F%252Fgithub.com%252Fmemphis-iis%252Fdatawhys-intern-notebooks-2023%26urlpath%3Dlab%252Ftree%252Fdatawhys-intern-notebooks-2023%252FDescriptive-statistics.ipynb%253Flog%253Dhttps%253A%252F%252Flogging2.olney.ai%252Fdatawhys%252Flog%2526bl%253Dpy%26branch%3Dmaster -> /hub/user/aolney/git-pull?repo=https%3A%2F%2Fgithub.com%2Fmemphis-iis%2Fdatawhys-intern-notebooks-2023&urlpath=lab%2Ftree%2Fdatawhys-intern-notebooks-2023%2FDescriptive-statistics.ipynb%3Flog%3Dhttps%3A%2F%2Flogging2.olney.ai%2Fdatawhys%2Flog%26bl%3Dpy&branch=master (aolney@10.101.244.254) 1.77ms Apr 23 19:27:05 datawhys python3[3169912]: [I 2024-04-23 19:27:05.603 JupyterHub log:192] 302 GET /hub/user/aolney/git-pull?repo=https%3A%2F%2Fgithub.com%2Fmemphis-iis%2Fdatawhys-intern-notebooks-2023&urlpath=lab%2Ftree%2Fdatawhys-intern-notebooks-2023%2FDescriptive-statistics.ipynb%3Flog%3Dhttps%3A%2F%2Flogging2.olney.ai%2Fdatawhys%2Flog%26bl%3Dpy&branch=master -> /user/aolney/git-pull?repo=https%3A%2F%2Fgithub.com%2Fmemphis-iis%2Fdatawhys-intern-notebooks-2023&urlpath=lab%2Ftree%2Fdatawhys-intern-notebooks-2023%2FDescriptive-statistics.ipynb%3Flog%3Dhttps%3A%2F%2Flogging2.olney.ai%2Fdatawhys%2Flog%26bl%3Dpy&branch=master&redirects=1 (aolney@10.101.244.254) 1.44ms ```
welcome[bot] commented 2 months ago

Thank you for opening your first issue in this project! Engagement like this is essential for open source projects! :hugs:
If you haven't done so already, check out Jupyter's Code of Conduct. Also, please try to follow the issue template as it helps other other community members to contribute more effectively. welcome You can meet the other Jovyans by joining our Discourse forum. There is also an intro thread there where you can stop by and say Hi! :wave:
Welcome to the Jupyter community! :tada:

consideRatio commented 2 months ago

I don't see a version of nbgitpuller used in old/new, and this is reported to jupyterhub/nbgitpuller, but you are using an nbgitpuller kind of link. What version have you been using before/after?

I'm not yet sure its a change in jupyterhub or nbgitpuller. To reduce the uncertainty here, can you instead of using /hub/user-redirect try the same thing with /user/yourusername` after having started your user server? If you do that, you should go straight to your user server without involvement from jupyterhub, so then we can rule out its a change in jupyterhub I think.

aolney commented 2 months ago

Thanks for the rapid response :)

1. Here are the versions (should I update the issue title?)

Is downgrading an option? Happy to give it a shot

2. Here's how I attempted to follow your suggestion

https://saturn.olney.ai/user/aolney/git-pull?repo=https%3A%2F%2Fgithub.com%2Fmemphis-iis%2Fdatawhys-intern-notebooks-2023&urlpath=lab%2Ftree%2Fdatawhys-intern-notebooks-2023%2FDescriptive-statistics.ipynb%3Flog%3Dhttps%3A%2F%2Flogging2.olney.ai%2Fdatawhys%2Flog%26bl%3Dpy&branch=master

This resulted in the same URL as before (for the new server):

https://saturn.olney.ai/user/aolney/lab/tree/datawhys-intern-notebooks-2023/Descriptive-statistics.ipynb
aolney commented 1 month ago

I tried downgrading to:

Here are the logs of it with 1.2.1:

May 07 19:54:47 datawhys python3[3923479]: [I 2024-05-07 19:54:47.406 JupyterHub log:192] 302 GET /hub/user-redirect/git-pull?repo=https%3A%2F%2Fgithub.com%2Fmemphis-iis%2Fdatawhys-intern-notebooks-2023&urlpath=lab%2Ftree%2Fdatawhys-intern-notebooks-2023%2FData-science-and-the-nature-of-data.ipynb%3Flog%3Dhttps%3A%2F%2Flogging2.olney.ai%2Fdatawhys%2Flog%26bl%3Dpy&branch=master -> /hub/spawn/aolney?next=%2Fuser%2Faolney%2Fgit-pull%3Frepo%3Dhttps%253A%252F%252Fgithub.com%252Fmemphis-iis%252Fdatawhys-intern-notebooks-2023%26urlpath%3Dlab%252Ftree%252Fdatawhys-intern-notebooks-2023%252FData-science-and-the-nature-of-data.ipynb%253Flog%253Dhttps%253A%252F%252Flogging2.olney.ai%252Fdatawhys%252Flog%2526bl%253Dpy%26branch%3Dmaster (aolney@10.228.101.5) 3.10ms
May 07 19:54:47 datawhys python3[3923479]: [W 2024-05-07 19:54:47.415 JupyterHub base:801] Redirecting /hub/spawn/aolney?next=%2Fuser%2Faolney%2Fgit-pull%3Frepo%3Dhttps%253A%252F%252Fgithub.com%252Fmemphis-iis%252Fdatawhys-intern-notebooks-2023%26urlpath%3Dlab%252Ftree%252Fdatawhys-intern-notebooks-2023%252FData-science-and-the-nature-of-data.ipynb%253Flog%253Dhttps%253A%252F%252Flogging2.olney.ai%252Fdatawhys%252Flog%2526bl%253Dpy%26branch%3Dmaster to /hub/user/aolney/git-pull?repo=https%3A%2F%2Fgithub.com%2Fmemphis-iis%2Fdatawhys-intern-notebooks-2023&urlpath=lab%2Ftree%2Fdatawhys-intern-notebooks-2023%2FData-science-and-the-nature-of-data.ipynb%3Flog%3Dhttps%3A%2F%2Flogging2.olney.ai%2Fdatawhys%2Flog%26bl%3Dpy&branch=master. For sharing public links, use /user-redirect/
May 07 19:54:47 datawhys python3[3923479]: [I 2024-05-07 19:54:47.415 JupyterHub log:192] 302 GET /hub/spawn/aolney?next=%2Fuser%2Faolney%2Fgit-pull%3Frepo%3Dhttps%253A%252F%252Fgithub.com%252Fmemphis-iis%252Fdatawhys-intern-notebooks-2023%26urlpath%3Dlab%252Ftree%252Fdatawhys-intern-notebooks-2023%252FData-science-and-the-nature-of-data.ipynb%253Flog%253Dhttps%253A%252F%252Flogging2.olney.ai%252Fdatawhys%252Flog%2526bl%253Dpy%26branch%3Dmaster -> /hub/user/aolney/git-pull?repo=https%3A%2F%2Fgithub.com%2Fmemphis-iis%2Fdatawhys-intern-notebooks-2023&urlpath=lab%2Ftree%2Fdatawhys-intern-notebooks-2023%2FData-science-and-the-nature-of-data.ipynb%3Flog%3Dhttps%3A%2F%2Flogging2.olney.ai%2Fdatawhys%2Flog%26bl%3Dpy&branch=master (aolney@10.228.101.5) 1.73ms
May 07 19:54:47 datawhys python3[3923479]: [I 2024-05-07 19:54:47.423 JupyterHub log:192] 302 GET /hub/user/aolney/git-pull?repo=https%3A%2F%2Fgithub.com%2Fmemphis-iis%2Fdatawhys-intern-notebooks-2023&urlpath=lab%2Ftree%2Fdatawhys-intern-notebooks-2023%2FData-science-and-the-nature-of-data.ipynb%3Flog%3Dhttps%3A%2F%2Flogging2.olney.ai%2Fdatawhys%2Flog%26bl%3Dpy&branch=master -> /user/aolney/git-pull?repo=https%3A%2F%2Fgithub.com%2Fmemphis-iis%2Fdatawhys-intern-notebooks-2023&urlpath=lab%2Ftree%2Fdatawhys-intern-notebooks-2023%2FData-science-and-the-nature-of-data.ipynb%3Flog%3Dhttps%3A%2F%2Flogging2.olney.ai%2Fdatawhys%2Flog%26bl%3Dpy&branch=master&redirects=1 (aolney@10.228.101.5) 1.41ms
May 07 19:57:01 datawhys python3[3923479]: [I 2024-05-07 19:57:01.321 JupyterHub log:192] 302 GET /hub/user-redirect/git-pull?repo=https%3A%2F%2Fgithub.com%2Fmemphis-iis%2Fdatawhys-intern-notebooks-2023&urlpath=lab%2Ftree%2Fdatawhys-intern-notebooks-2023%2FData-science-and-the-nature-of-data.ipynb%3Flog%3Dhttps%3A%2F%2Flogging2.olney.ai%2Fdatawhys%2Flog%26bl%3Dpy&branch=master -> /hub/spawn/aolney?next=%2Fuser%2Faolney%2Fgit-pull%3Frepo%3Dhttps%253A%252F%252Fgithub.com%252Fmemphis-iis%252Fdatawhys-intern-notebooks-2023%26urlpath%3Dlab%252Ftree%252Fdatawhys-intern-notebooks-2023%252FData-science-and-the-nature-of-data.ipynb%253Flog%253Dhttps%253A%252F%252Flogging2.olney.ai%252Fdatawhys%252Flog%2526bl%253Dpy%26branch%3Dmaster (aolney@10.228.101.5) 3.12ms
May 07 19:57:01 datawhys python3[3923479]: [W 2024-05-07 19:57:01.330 JupyterHub base:801] Redirecting /hub/spawn/aolney?next=%2Fuser%2Faolney%2Fgit-pull%3Frepo%3Dhttps%253A%252F%252Fgithub.com%252Fmemphis-iis%252Fdatawhys-intern-notebooks-2023%26urlpath%3Dlab%252Ftree%252Fdatawhys-intern-notebooks-2023%252FData-science-and-the-nature-of-data.ipynb%253Flog%253Dhttps%253A%252F%252Flogging2.olney.ai%252Fdatawhys%252Flog%2526bl%253Dpy%26branch%3Dmaster to /hub/user/aolney/git-pull?repo=https%3A%2F%2Fgithub.com%2Fmemphis-iis%2Fdatawhys-intern-notebooks-2023&urlpath=lab%2Ftree%2Fdatawhys-intern-notebooks-2023%2FData-science-and-the-nature-of-data.ipynb%3Flog%3Dhttps%3A%2F%2Flogging2.olney.ai%2Fdatawhys%2Flog%26bl%3Dpy&branch=master. For sharing public links, use /user-redirect/
May 07 19:57:01 datawhys python3[3923479]: [I 2024-05-07 19:57:01.330 JupyterHub log:192] 302 GET /hub/spawn/aolney?next=%2Fuser%2Faolney%2Fgit-pull%3Frepo%3Dhttps%253A%252F%252Fgithub.com%252Fmemphis-iis%252Fdatawhys-intern-notebooks-2023%26urlpath%3Dlab%252Ftree%252Fdatawhys-intern-notebooks-2023%252FData-science-and-the-nature-of-data.ipynb%253Flog%253Dhttps%253A%252F%252Flogging2.olney.ai%252Fdatawhys%252Flog%2526bl%253Dpy%26branch%3Dmaster -> /hub/user/aolney/git-pull?repo=https%3A%2F%2Fgithub.com%2Fmemphis-iis%2Fdatawhys-intern-notebooks-2023&urlpath=lab%2Ftree%2Fdatawhys-intern-notebooks-2023%2FData-science-and-the-nature-of-data.ipynb%3Flog%3Dhttps%3A%2F%2Flogging2.olney.ai%2Fdatawhys%2Flog%26bl%3Dpy&branch=master (aolney@10.228.101.5) 1.34ms
May 07 19:57:01 datawhys python3[3923479]: [I 2024-05-07 19:57:01.338 JupyterHub log:192] 302 GET /hub/user/aolney/git-pull?repo=https%3A%2F%2Fgithub.com%2Fmemphis-iis%2Fdatawhys-intern-notebooks-2023&urlpath=lab%2Ftree%2Fdatawhys-intern-notebooks-2023%2FData-science-and-the-nature-of-data.ipynb%3Flog%3Dhttps%3A%2F%2Flogging2.olney.ai%2Fdatawhys%2Flog%26bl%3Dpy&branch=master -> /user/aolney/git-pull?repo=https%3A%2F%2Fgithub.com%2Fmemphis-iis%2Fdatawhys-intern-notebooks-2023&urlpath=lab%2Ftree%2Fdatawhys-intern-notebooks-2023%2FData-science-and-the-nature-of-data.ipynb%3Flog%3Dhttps%3A%2F%2Flogging2.olney.ai%2Fdatawhys%2Flog%26bl%3Dpy&branch=master&redirects=1 (aolney@10.228.101.5) 1.85ms

Any suggestions of other things I can try?

aolney commented 1 month ago

We seem to be able to solve this for our use case, so I'm closing. If anyone else has this problem, our solution is to have the extension always activated, in which case it can grab the query string parameters before they are scrubbed.