search

jupyterhub / jupyter-rsession-proxy

Jupyter extensions for running an RStudio rsession proxy
BSD 3-Clause "New" or "Revised" License
118 stars 87 forks source link

What port will this run on? #14

Open arokem opened 6 years ago

arokem commented 6 years ago

I am trying to install this on AWS, on top of a working Jupyterhub installation. Does this require opening access through another port? I noticed that port 46645 is mentioned in the Jupyterhub logs. Should I create a security group with that port open, in addition to ports I have open for Jupyterhub? I am getting a 404 when I run this without that port open.

ryanlovett commented 6 years ago

This doesn't require opening additional ports. The nbserverproxy extension proxies to arbitrary web servers through the users Jupyter server. nbrsessionproxy launches RStudio on a random port, then uses nbserverproxy to expose that to the user's browser. See https://github.com/jupyterhub/nbserverproxy/ for a brief description.

arokem commented 6 years ago

OK. Maybe I will describe to you what I did, and what I am seeing and you can help me figure out why this is not working. I am working on an AWS ubuntu machine. I started by installing miniconda, and used that to install jupyterhub. I configure that and that all seems to work fine -- I can run Python notebooks without any problem (I had to install xorg to run R notebooks, as described here).

Then, I used apt-get and conda to install all of these things: https://github.com/jupyter/docker-stacks/blob/master/r-notebook/Dockerfile#L10-L37. I also ran conda install rstudio, because that doesn't seem to be installed in these instructions. I then pip install nbserverproxy and nbrsessionproxy and enable them, as described in the README for these two.

When I run sudo jupyterhub --config yada_yada.py, I get (I set debug_proxy to True, so it's rather verbose): 


sudo jupyterhub --config jupyterhub_config.py 
[I 2017-10-13 03:15:05.144 JupyterHub app:724] Loading cookie_secret from /home/ubuntu/jupyterhub_cookie_secret
[W 2017-10-13 03:15:05.178 JupyterHub app:365] 
    Generating CONFIGPROXY_AUTH_TOKEN. Restarting the Hub will require restarting the proxy.
    Set CONFIGPROXY_AUTH_TOKEN env or JupyterHub.proxy_auth_token config to avoid this message.

[W 2017-10-13 03:15:05.181 JupyterHub app:864] No admin users, admin interface will be unavailable.
[W 2017-10-13 03:15:05.181 JupyterHub app:865] Add any administrative users to `c.Authenticator.admin_users` in config.
[I 2017-10-13 03:15:05.181 JupyterHub app:892] Not using whitelist. Any authenticated user will be allowed.
[I 2017-10-13 03:15:05.200 JupyterHub app:1453] Hub API listening on http://127.0.0.1:8081/hub/
[I 2017-10-13 03:15:05.203 JupyterHub app:1176] Starting proxy @ http://*:443/
03:15:05.331 - info: [ConfigProxy] Proxying https://*:443 to http://127.0.0.1:8081
03:15:05.334 - info: [ConfigProxy] Proxy API at http://127.0.0.1:444/api/routes
[I 2017-10-13 03:15:05.408 JupyterHub app:1485] JupyterHub is now running at http://127.0.0.1:443/
03:15:13.761 - debug: [ConfigProxy] PROXY WEB / to http://127.0.0.1:8081
[I 2017-10-13 03:15:13.777 JupyterHub log:100] 302 GET / (@::ffff:97.113.100.107) 1.35ms
03:15:13.792 - debug: [ConfigProxy] PROXY WEB /hub to http://127.0.0.1:8081
[I 2017-10-13 03:15:13.795 JupyterHub log:100] 302 GET /hub (@::ffff:97.113.100.107) 0.39ms
03:15:13.811 - debug: [ConfigProxy] PROXY WEB /hub/ to http://127.0.0.1:8081
[I 2017-10-13 03:15:13.818 JupyterHub log:100] 302 GET /hub/ (arokem@::ffff:97.113.100.107) 2.21ms
03:15:13.835 - debug: [ConfigProxy] PROXY WEB /hub/home to http://127.0.0.1:8081
[I 2017-10-13 03:15:13.865 JupyterHub log:100] 200 GET /hub/home (arokem@::ffff:97.113.100.107) 25.85ms
03:15:13.898 - debug: [ConfigProxy] PROXY WEB /hub/static/css/style.min.css?v=d96e0760e0c2b7356ce89635b646c350 to http://127.0.0.1:8081
03:15:13.952 - debug: [ConfigProxy] PROXY WEB /hub/logo to http://127.0.0.1:8081
03:15:14.267 - debug: [ConfigProxy] PROXY WEB /hub/static/components/requirejs/require.js?v=6da8be361b9ee26c5e721e76c6d4afce to http://127.0.0.1:8081
03:15:14.314 - debug: [ConfigProxy] PROXY WEB /hub/static/components/font-awesome/fonts/fontawesome-webfont.woff?v=4.1.0 to http://127.0.0.1:8081
03:15:14.340 - debug: [ConfigProxy] PROXY WEB /hub/static/js/home.js?v=(& to http://127.0.0.1:8081
03:15:14.457 - debug: [ConfigProxy] PROXY WEB /hub/static/components/jquery/jquery.min.js?v=(& to http://127.0.0.1:8081
03:15:14.483 - debug: [ConfigProxy] PROXY WEB /hub/static/js/jhapi.js?v=(& to http://127.0.0.1:8081
03:15:14.538 - debug: [ConfigProxy] PROXY WEB /hub/static/js/utils.js?v=(& to http://127.0.0.1:8081
03:15:15.903 - debug: [ConfigProxy] PROXY WEB /hub/spawn to http://127.0.0.1:8081
[I 2017-10-13 03:15:15.911 JupyterHub log:100] 302 GET /hub/spawn (arokem@::ffff:97.113.100.107) 3.21ms
03:15:15.924 - debug: [ConfigProxy] PROXY WEB /user/arokem to http://127.0.0.1:8081
[I 2017-10-13 03:15:15.929 JupyterHub log:100] 302 GET /user/arokem (@::ffff:97.113.100.107) 0.81ms
03:15:15.940 - debug: [ConfigProxy] PROXY WEB /hub/user/arokem to http://127.0.0.1:8081
Last login: Fri Oct 13 03:11:30 UTC 2017
Welcome to Ubuntu 16.04.2 LTS (GNU/Linux 4.4.0-1038-aws x86_64)

 * Documentation:  https://help.ubuntu.com
 * Management:     https://landscape.canonical.com
 * Support:        https://ubuntu.com/advantage

  Get cloud support with Ubuntu Advantage Cloud Guest:
    http://www.ubuntu.com/business/services/cloud

88 packages can be updated.
0 updates are security updates.

[I 2017-10-13 03:15:16.028 JupyterHub spawner:783] Spawning jupyterhub-singleuser '--user="arokem"' '--cookie-name="jupyter-hub-token-arokem"' '--base-url="/user/arokem"' '--hub-host=""' '--hub-prefix="/hub/"' '--hub-api-url="http://127.0.0.1:8081/hub/api"' '--ip="127.0.0.1"' --port=44997
[W 2017-10-13 03:15:17.062 arokem login:225] All authentication is disabled.  Anyone who can connect to this server will be able to run code.
[I 2017-10-13 03:15:17.071 arokem notebookapp:1366] Serving notebooks from local directory: /home/arokem
[I 2017-10-13 03:15:17.071 arokem notebookapp:1366] 0 active kernels 
[I 2017-10-13 03:15:17.071 arokem notebookapp:1366] The Jupyter Notebook is running at: http://127.0.0.1:44997/user/arokem/
[I 2017-10-13 03:15:17.071 arokem notebookapp:1367] Use Control-C to stop this server and shut down all kernels (twice to skip confirmation).
[I 2017-10-13 03:15:17.158 arokem log:47] 302 GET /user/arokem (127.0.0.1) 0.56ms
[I 2017-10-13 03:15:17.159 JupyterHub base:322] User arokem server took 1.214 seconds to start
[I 2017-10-13 03:15:17.160 JupyterHub orm:188] Adding user arokem to proxy /user/arokem => http://127.0.0.1:44997
03:15:17.164 - debug: [ConfigProxy] POST /user/arokem target=http://127.0.0.1:44997, user=arokem
[I 2017-10-13 03:15:17.172 JupyterHub log:100] 302 GET /hub/user/arokem (arokem@::ffff:97.113.100.107) 1227.36ms
03:15:17.202 - debug: [ConfigProxy] PROXY WEB /user/arokem to http://127.0.0.1:44997
[I 2017-10-13 03:15:17.204 arokem log:47] 302 GET /user/arokem (::ffff:97.113.100.107) 0.45ms
03:15:17.219 - debug: [ConfigProxy] PROXY WEB /user/arokem/tree? to http://127.0.0.1:44997
[I 2017-10-13 03:15:17.246 JupyterHub log:100] 200 GET /hub/api/authorizations/cookie/jupyter-hub-token-arokem/[secret] (arokem@127.0.0.1) 11.18ms
03:15:17.333 - debug: [ConfigProxy] PROXY WEB /user/arokem/static/components/jquery-ui/themes/smoothness/jquery-ui.min.css?v=9b2c8d3489227115310662a343fce11c to http://127.0.0.1:44997
03:15:17.358 - debug: [ConfigProxy] PROXY WEB /user/arokem/static/style/style.min.css?v=29c09309dd70e7fe93378815e5f022ae to http://127.0.0.1:44997
03:15:17.575 - debug: [ConfigProxy] PROXY WEB /user/arokem/static/components/jquery-typeahead/dist/jquery.typeahead.min.css?v=7afb461de36accb1aa133a1710f5bc56 to http://127.0.0.1:44997
03:15:17.596 - debug: [ConfigProxy] PROXY WEB /user/arokem/custom/custom.css to http://127.0.0.1:44997
03:15:17.600 - debug: [ConfigProxy] PROXY WEB /user/arokem/static/components/preact/index.js?v=5b98fce8b86ce059de89f9e728e16957 to http://127.0.0.1:44997
03:15:17.600 - debug: [ConfigProxy] PROXY WEB /user/arokem/static/components/es6-promise/promise.min.js?v=f004a16cb856e0ff11781d01ec5ca8fe to http://127.0.0.1:44997
03:15:17.603 - debug: [ConfigProxy] PROXY WEB /user/arokem/static/components/proptypes/index.js?v=c40890eb04df9811fcc4d47e53a29604 to http://127.0.0.1:44997
03:15:17.605 - debug: [ConfigProxy] PROXY WEB /user/arokem/static/components/preact-compat/index.js?v=d376eb109a00b9b2e8c0d30782eb6df7 to http://127.0.0.1:44997
03:15:17.607 - debug: [ConfigProxy] PROXY WEB /user/arokem/static/components/requirejs/require.js?v=6da8be361b9ee26c5e721e76c6d4afce to http://127.0.0.1:44997
03:15:17.642 - debug: [ConfigProxy] PROXY WEB /user/arokem/static/tree/js/main.min.js?v=9a7bc408ac5c5ba9cc978f3eea1bb741 to http://127.0.0.1:44997
03:15:17.795 - debug: [ConfigProxy] PROXY WEB /user/arokem/static/services/contents.js?v=20171013031516 to http://127.0.0.1:44997
03:15:17.839 - debug: [ConfigProxy] PROXY WEB /user/arokem/api/config/tree?_=1507864517811 to http://127.0.0.1:44997
03:15:17.843 - debug: [ConfigProxy] PROXY WEB /user/arokem/api/config/common?_=1507864517812 to http://127.0.0.1:44997
03:15:17.851 - debug: [ConfigProxy] PROXY WEB /user/arokem/api/terminals?_=1507864517813 to http://127.0.0.1:44997
03:15:17.858 - debug: [ConfigProxy] PROXY WEB /user/arokem/api/kernelspecs to http://127.0.0.1:44997
03:15:17.866 - debug: [ConfigProxy] PROXY WEB /user/arokem/api/sessions?_=1507864517814 to http://127.0.0.1:44997
03:15:17.867 - debug: [ConfigProxy] PROXY WEB /user/arokem/api/terminals?_=1507864517815 to http://127.0.0.1:44997
03:15:17.874 - debug: [ConfigProxy] PROXY WEB /user/arokem/static/components/font-awesome/fonts/fontawesome-webfont.woff?v=4.2.0 to http://127.0.0.1:44997
03:15:18.023 - debug: [ConfigProxy] PROXY WEB /user/arokem/api/contents?type=directory&_=1507864517816 to http://127.0.0.1:44997
03:15:18.027 - debug: [ConfigProxy] PROXY WEB /user/arokem/custom/custom.js?v=20171013031516 to http://127.0.0.1:44997
03:15:18.040 - debug: [ConfigProxy] PROXY WEB /user/arokem/nbextensions/nbrsessionproxy/tree.js?v=20171013031516 to http://127.0.0.1:44997
03:15:21.024 - debug: [ConfigProxy] PROXY WEB /user/arokem/rsessionproxy to http://127.0.0.1:44997
sleeping: [Errno 111] Connection refused
03:15:23.361 - debug: [ConfigProxy] PROXY WEB /user/arokem/proxy/40031/ to http://127.0.0.1:44997
[I 2017-10-13 03:15:23.363 arokem log:47] 302 GET /user/arokem/proxy/40031/ (::ffff:97.113.100.107) 0.60ms
03:15:23.378 - debug: [ConfigProxy] PROXY WEB /user/arokem/proxy/40031 to http://127.0.0.1:44997
[W 2017-10-13 03:15:23.387 arokem log:47] 404 GET /user/arokem/proxy/40031 (::ffff:97.113.100.107) 7.48ms referer=https://ec2-54-245-61-246.us-west-2.compute.amazonaws.com/user/arokem/tree?
03:15:23.440 - debug: [ConfigProxy] PROXY WEB /user/arokem/static/components/jquery-ui/themes/smoothness/jquery-ui.min.css?v=9b2c8d3489227115310662a343fce11c to http://127.0.0.1:44997
03:15:23.443 - debug: [ConfigProxy] PROXY WEB /user/arokem/static/components/jquery-typeahead/dist/jquery.typeahead.min.css?v=7afb461de36accb1aa133a1710f5bc56 to http://127.0.0.1:44997
03:15:23.443 - debug: [ConfigProxy] PROXY WEB /user/arokem/static/style/style.min.css?v=29c09309dd70e7fe93378815e5f022ae to http://127.0.0.1:44997
03:15:23.444 - debug: [ConfigProxy] PROXY WEB /user/arokem/custom/custom.css to http://127.0.0.1:44997
03:15:23.567 - debug: [ConfigProxy] PROXY WEB /user/arokem/static/components/jquery/jquery.min.js?v=20171013031516 to http://127.0.0.1:44997

In the attached browser, I get the Jupyter 404 page.

Do you understand what [Errno 111] Connection refused means here? Seems like this could be the crux of the problem, and maybe this thread offers something useful, but I don't understand it. Does it ring any bells? Thanks!

ryanlovett commented 6 years ago

Can you please include a screenshot of the error page?

Does the nbserverproxy example work for you? For example, in a terminal type:

mkdir ~/www
cd ~/www
echo hello > index.html
python -m http.server 9000

then navigate to your.hub.com/user/aroken/proxy/9000.

If you run a Jupyter terminal, can you tell if your RStudio process is running in the background? (for example ps aux | grep -i rstudio)

It sounds as if RStudio did not start properly so the proxy service has nothing to proxy to.

It may be necessary to configure PATH and LD_LIBRARY_PATH for your environment. On Ubuntu, one needs to append /usr/lib/rstudio-server/bin to PATH and set LD_LIBRARY_PATH to find R shared libraries. See the end of the Dockerfile in this repo.

PATH="${PATH}:/usr/lib/rstudio-server/bin"
LD_LIBRARY_PATH="/usr/lib/R/lib:/lib:/usr/lib/x86_64-linux-gnu:/usr/lib/jvm/java-7-openjdk-amd64/jre/lib/amd64/server:/opt/conda/lib/R/lib"

If you set PATH and LD_LIBRARY_PATH before invoking sudo jupyterhub..., you can set c.Spawner.env_keep.append('PATH') and c.Spawner.env_keep.append('LD_LIBRARY_PATH') in your JupyterHub config file.

arokem commented 6 years ago

Thanks a lot for all your thoughts about this!

Even the simple case doesn't seem to work for me. This is the 404 page I am seeing, running the snippet of code you wrote:

screen shot 2017-10-13 at 3 07 52 pm

On the bright side (?), ps aux | grep -i rstudio does find an rstudio process that is running, when I try to open that window. Again, I see a very similar 404 page for that, but proxying a different port:

screen shot 2017-10-13 at 9 06 47 pm

ryanlovett commented 6 years ago

This sounds like there's a problem with nbserverproxy. Can you run jupyter serverextension list ?

If that shows that nbserverproxy is enabled and OK, can you please open the JavaScript console in your browser (Three dots > More Tools > Developer Tools ; or type Command-Option-I). Click the "Console" tab and check for errors pertaining to nbserverproxy.

arokem commented 6 years ago

The extension seems to be enabled and OK:


ubuntu@ip-172-31-43-211:~$ jupyter serverextension list
config dir: /home/ubuntu/.jupyter
    nbserverproxy  enabled 
    - Validating...
      nbserverproxy  OK
    nbrsessionproxy  enabled 
    - Validating...
      nbrsessionproxy  OK
config dir: /opt/anaconda/etc/jupyter
    nbrsessionproxy  enabled 
    - Validating...
      nbrsessionproxy  OK

In the JS console, the only error that appears seems to be unrelated: 


TypeError: Cannot read property 'scrollHeight' of undefined
    at 9000:181
    at Object.execCb (require.js?v=6da8be361b9ee26c5e721e76c6d4afce:1690)
    at Module.check (require.js?v=6da8be361b9ee26c5e721e76c6d4afce:865)
    at Module.<anonymous> (require.js?v=6da8be361b9ee26c5e721e76c6d4afce:1140)
    at require.js?v=6da8be361b9ee26c5e721e76c6d4afce:131
    at require.js?v=6da8be361b9ee26c5e721e76c6d4afce:1190
    at each (require.js?v=6da8be361b9ee26c5e721e76c6d4afce:56)
    at Module.emit (require.js?v=6da8be361b9ee26c5e721e76c6d4afce:1189)
    at Module.check (require.js?v=6da8be361b9ee26c5e721e76c6d4afce:940)
    at Module.enable (require.js?v=6da8be361b9ee26c5e721e76c6d4afce:1177)
    at Module.init (require.js?v=6da8be361b9ee26c5e721e76c6d4afce:783)
    at callGetModule (require.js?v=6da8be361b9ee26c5e721e76c6d4afce:1204)
    at Object.completeLoad (require.js?v=6da8be361b9ee26c5e721e76c6d4afce:1583)
    at HTMLScriptElement.onScriptLoad (require.js?v=6da8be361b9ee26c5e721e76c6d4afce:1711)
ryanlovett commented 6 years ago

It appears that nbserverproxy is not installed in the anaconda path. Can you invoke pip and jupyter as the user that owns /opt/anaconda when installing nbserverproxy?

iagomez commented 6 years ago

@arokem Depending on your setup you may want to enable this extension for all users. The README for nbserverproxy will set it up for the user executing the pip command.

You can set it up system-wide this way:

pip install git+https://github.com/jupyterhub/nbserverproxy
jupyter serverextension enable --py --sys-prefix nbserverproxy

Hope this helps!

riedel commented 4 years ago

Does this mean that by guessing the port number I can access another rstudio session? How is security enforced by the proxy when running it on a hub (that does not isolate the users networking)?

Jupyter is using tokens for this, but this seems to be not the case for rsession in standalone mode (reading the code and trying out the command from the code)

(This remotely related to the ticket, but the title is fitting and I don't want to waste a ticket on sth that is not really a bug)

ryanlovett commented 4 years ago

@riedel If users' jupyter and rstudio processes are running on a shared system, not namespaced or containerized in any way, then one user may be able to access another user's rstudio session.

While user1 cannot access /user2/rstudio because it uses jupyter's authentication, user1 can access /user1/proxy/N where N is the port on which user2's rstudio or any other listening process is running.