X-Forwarded-Proto header causes 404 with rstudio server 1.4

[ryanlovett]

As mentioned in #95.

If the HTTP header X-Forwarded-Proto is being set, such as by a proxy, rstudio-server will redirect users to /auth-sign-in?appUri=%2F rather than the full {base_url}/auth-sign-in?appUri=%2F. This will usually cause a 404. You can observe this behavior on mybinder.org for example, whereas it probably is not present on localhost.

I created https://github.com/rstudio/rstudio/issues/8888, however I don't see anything in rstudio's code that would cause this behavior. Perhaps we're not invoking rserver with the right combination of options.

When presented with the 404, a user can manually work around the problem by inserting the base path in front of the auth-sign-in path. For example on a jupyterhub, the user could change /auth-sign-in?appUri=%2F to /user/the-user-name/rstudio/auth-sign-in?appUri=%2F. Once corrected, rserver will properly authenticate the user and the user will be redirected to {base_url}/rstudio. Obviously we don't want the user to have to do this, but this observation helps to describe the problem.

If we remove the header from the HTTP request, the Location header returned by rstudio server uses http protocol and has the correct path. So this might not be appropriate on https sites. If we intercede and set the value of the header to https instead of https,http, the header value uses https:// rather than http://.

The argument against filtering out the header is that if rserver ever requires https, not having the header could lead to a redirect loop. I found a nice explanation of this.

@manics also described past experience with the header.

We might be able to workaround this issue by setting our url to /rstudio/auth-sign-in?appUri=%2F instead of /rstudio. If you've already been authenticated by rserver, then this just takes the user back to /rstudio/. I'd have to experiment but I think it'd work just fine.

[vnijs]

@ryanlovett Is it possible to turn off authentication in Rstudio as used to be possible to get around this issue? I rely on jupyterhub for authentication so don't need it from Rstudio as well.

[ryanlovett]

@vnijs We do set rstudio's --auth-none flag and one doesn't need to actually authenticate to rstudio, but rserver initiates this redirect flow anyways. We only notice because it is sending the user to the wrong place.

[vnijs]

Is this perhaps something to create an issue about at https://github.com/rstudio/rstudio/issues? I might also be able to check with Rstudio support directly if you think that would be a better option.

[ryanlovett]

@vnijs Yes, see https://github.com/rstudio/rstudio/issues/8888.

[meeseeksmachine]

This issue has been mentioned on Jupyter Community Forum. There might be relevant details there:

https://discourse.jupyter.org/t/jupyter-server-proxy-setup-for-rstudio/8333/2

[ocaisa]

For me, the potential workaround mentioned in the first post leads to ERR_TOO_MANY_REDIRECTS

[riazarbi]

Hello,

Carrying on my investigations from #93, with the flag RSESSION_PROXY_RSTUDIO_1_4 set and RStudio server preview release 1.4.1722.

I've forked this repo and am messing around on my fork here -

https://github.com/jupyterhub/jupyter-rsession-proxy/compare/master...riazarbi:patch-1

I've changed the line

cmd.append('--www-root-path={base_url}rstudio/')

to

cmd.append('--www-root-path={base_url}rstudio/auth-sign-in?appUri=%2F'

and rebuilt my image and attempted to launch Rstudio from the lab launcher.

The result is the same whether I launch via localhost docker or jupyterhub kubernetes. I get redirected to this uri.

http://localhost:8888/rstudio/auth-sign-in?appUri=%2F/./

or

https://datascience.capetown.gov.za/workbench/user/rarbi/rstudio/auth-sign-in?appUri=%2F/./

Respectively. Next I'll try this approach - https://github.com/rstudio/rstudio/issues/2834#issuecomment-777097130

And see if it helps.

[riazarbi]

Building my image with the line

cmd.append('--www-root-path={base_url}rstudio/')

changed to

cmd.append('--www-root-path={base_url}rstudio/auth-sign-in?appUri=%2Frstudio')

Results -

On localhost, clicking launcher takes me to -

http://127.0.0.1:8888/rstudio/auth-sign-in?appUri=%2Frstudio/./

On jupyterhub, clicking launcher takes me to -

https://datascience.capetown.gov.za/workbench/user/rarbi/rstudio/auth-sign-in?appUri=%2Frstudio/./

Looking at the Rstudio source code, the /./ is being appended here -

https://github.com/rstudio/rstudio/blob/51009f4cff7b0f45f9bd8762d7c0514b0de537bd/src/cpp/server/auth/ServerAuthCommon.cpp#L90

Basically, it looks to me (a lowly data scientist) that the appUri is not being set.

      if (appUri.empty() || appUri[0] != '/')
      {
         appUri = "./" + appUri;
      }

I've noticed a pull request from @zeehio that just came through; off the back of that I'll try rstudio/auth-sign-in and see if it works.

[riazarbi]

cmd.append('--www-root-path={base_url}rstudio/auth-sign-in') takes me to http://localhost:8888/rstudio/auth-sign-in/ and a redirect loop failure on localhost and https://datascience.capetown.gov.za/auth-sign-in?appUri=%2F and 404 not found on kubernetes.

I have not tested @zeehio 's solution

[ryanlovett]

As of v1.4 of this extension, we are working around this issue. I'd like to leave it open to help track the upstream progress. I think we'll close this when the workaround is no longer necessary.

[Robinlovelace]

Great to hear of progress, many thanks Ryan. I would be interested in seeing details of the work-around and ideas on making the work-around unnecessary, especially as I am hoping to use Binder/RStudio for teaching from September 2021.

[zeehio]

The workaround is fairly simple:

When you visit /rstudio this happens:

• Are you authenticated against rstudio?
  • yes -> RStudio opens
  • no -> You are redirected to the auth-sign-in page. <- This redirection does not work on v1.4

If you visit the /rstudio/auth-sign-in page the following happens:

• Are you authenticated against rstudio (or is RStudio authentication disabled)?
  • yes -> Redirected to /rstudio and Rstudio Opens
  • no -> the login page appears

Before our workaround, the only way to open rstudio successfully was to manually type and visit the auth-sign-in endpoint because of the broken redirection in /rstudio.

After our workaround, the RStudio button in jupyter will always take you to the correct auth-sign-in page. We have always had the RStudio authentication disabled here (because auth has been taken care by jupyter already) so you won't ever see the login page and you will be redirected to RStudio transparently.

The workaround won't be necessary if the RStudio folks fix the issue. However it is not a big deal to maintain it, in my opinion.

[JamesSample]

I'm having trouble getting the workaround described by @zeehio to function on my JupyterHub. Does anyone have any tips, please? (The Hub is running on Google Cloud and it's based on the Zero2JupyterHub tutorial).

I was originally experiencing the problem exactly as described @ryanlovett in the first post of this thread. The issue initially occurred both on my cloud-based JupyterHub and when testing locally using Docker (i.e. launching a container and accessing JupyterHub on localhost).

Upgrading jupyter-rsession-proxy to version 1.4 and adding ENV RSESSION_PROXY_RSTUDIO_1_4=1 to my Dockerfile solved the problem when running locally, but when I deploy the same container on my cloud-based JupyterHub I still experience the original problem: when I click the R-Studio icon in JupyterLab I'm redirected to

https://myjupyterhub.com/hub/auth-sign-in?appUri=%2F

and see a 404 error. If I manually enter the URL as

https://myjupyterhub.com/user/jamessample/rstudio/auth-sign-in?appUri=%2F

everything works fine for the rest of the session, but I'd like to avoid asking all my users to do this each time if possible.

Upgrading to 1.4 and adding the environment variable seems to have almost solved the problem, but there's still something I'm missing. Can anyone point me in the right direction, please?

Thanks!

[zeehio]

As far as I can recall in the pull requests that were merged it is mentioned that the workaround currently works for the jupyter menu, but it does not work for the jupyterlab icons because we don't have the JavaScript experience to fix it there

[JamesSample]

Ah, OK - thanks for the lightning fast response @zeehio!

Interesting that it fixes the problem with JupyterLab running locally, though. I'll take a look at the pull requests for more details.

Cheers!

[erniee]

I had the same issue, but I have it working now on my Zero to Jupyterhub GKE cluster. Would you mind checking my work to keep me honest? I did the following: IN MY DOCKERFILE

1. Used jupyter/r-notebook:r-4.1.0 as my base image
2. Installed rstudio-server version 1.4.1717
3. Added ENV RSESSION_PROXY_RSTUDIO_1_4=yes

IN MY CONFIG.YAML I added the image with the defaultUrl specified as: defaultUrl: "/rstudio/auth-sign-in?appUri=%2F"

Github link to my Dockerfile repo

[JamesSample]

@erniee I haven't tested your Dockerfile on my cluster, but I think your solution will work for anyone wanting to start R-Studio by default 👍

In my case, I'd like to use JupyterLab by default, but then have the option of starting R-Studio by clicking the button in the JupyterLab launcher menu. If you change your config.yaml to use defaultUrl: "/lab" instead and then start R-Studio from the button, do you still see a 404 error?

Great that it's working for your use case though!

[ryanlovett]

Hi all,

Could you please try out https://github.com/jupyterhub/jupyter-rsession-proxy/pull/107 if convenient? It should work in both lab and notebook as it doesn't rely on the modified launcher. You should be able to pip install it. If you do try it, please let me know how it goes on that PR. I've tested it locally and mybinder.

[JamesSample]

This is working now for me - please see my comment on the PR.

Many thanks @ryanlovett!

[arghya18]

I was facing the same issue in z2jh on k8s and its working now with below config in Ingress

nginx.ingress.kubernetes.io/configuration-snippet: |
      if ( $http_referer ~ ".*user\/(?<user_id>[\w-]+)\/lab$" ) {
        rewrite ^/(?<arg_suffix>auth-sign-in.*) "https://$server_name/user/$user_id/rstudio/$arg_suffix";
      }

[meeseeksmachine]

This issue has been mentioned on Jupyter Community Forum. There might be relevant details there:

https://discourse.jupyter.org/t/psa-latest-rstudio-will-not-work-with-jupyterhub-or-binder-use-r-version-4-0/10957/1

[ryanlovett]

Addressed by #110. We'll make a new release shortly.