search

jupyterhub / jupyter-server-proxy

Jupyter notebook server extension to proxy web services.
https://jupyter-server-proxy.readthedocs.io
BSD 3-Clause "New" or "Revised" License
348 stars 148 forks source link

Usage with `dask-labextension` #70

Open ian-r-rose opened 5 years ago

ian-r-rose commented 5 years ago

This is a follow-up with from some gitter chatting with @yuvipanda.

The dask-labextension JupyterLab extension currently involves adding iframe'd bokeh dashboard panels to the main work area. In order to get around CORS issues, it includes a modified version of nbserverproxy to proxy them under the notebook server origin. It is possible to do the same thing with an unmodified version of the package, but there were a couple of things that made me include the modified version:

  1. I would like to leave open the possibility that the dashboard URL is not on localhost. Our current examples are all on localhost, but I don't view that as a requirement that needs to be enforced, and we want to be able to support a wide variety of dask clusters which may have different deployment patterns that we haven't necessarily thought of.
  2. The dask labextension includes rest endpoints for cluster management as well as dashboard proxying. Currently, the cluster management endpoint is at /dask/clusters/<cluster-id>, and the corresponding dashboard is at /dask/dashboards/<cluster-id>. I'd like to be able to continue this pattern.

So there are two things that would be nice for nbserverproxy (or jupyter-server-proxy) to allow to be configured:

  1. The ability to proxy to arbitrary URLs, rather than just ports on localhost.
  2. The ability to place a given proxy at a different endpoint on the notebook server than /proxy/<port>/<path>.

I know that @yuvipanda has put some significant work into revamping this package recently, so it is possible that one can already do this. If that's the case, do let me know!

ryanlovett commented 5 years ago

@jacobtomlinson has a patch to address the first item on your wish list at https://github.com/jupyterhub/nbserverproxy/pull/46.

You can already do the second item. See how nbstencila accomplishes it in the latest release and how nbrsessionproxy does it in master.

ian-r-rose commented 5 years ago

Thanks for the info @ryanlovett! I'll do some experiments with the current overhauled version and see how far I can get.

yuvipanda commented 5 years ago

@ian-r-rose hold on for a day or two, and I think it'll be clearer.

ian-r-rose commented 5 years ago

Any further thoughts on this @yuvipanda? The linked examples seem to be more oriented around traitlets and entry points. I'd probably prefer to drive it programmatically rather than via configuration in my use case, however.

yuvipanda commented 5 years ago

@ian-r-rose hmm, can you explain what you mean by 'programmatically'? Would you like an HTTP API? Or a pythonic API that you can dynamically call from other extensions, instead of just entrypoints?

ian-r-rose commented 5 years ago

I mean more of the second. In my case, I am starting dask clusters from python code in a server extension. From them, I know the dashboard url, and I know the URL to which I want to proxy them. So they are not really a config option or an entrypoint; instead, I want to dynamically set up the proxies.

ian-r-rose commented 5 years ago

@yuvipanda I'm starting to think about this again, and would love to stop using my fork of the earlier nbserverproxy. If you were able to provide some guidance/review, I'd be happy to start putting some cycles in to doing the actual coding.

FYI, the crux of the extra logic in my fork is here: https://github.com/dask/dask-labextension/blob/910f5fff9e4d50f79a8c42e0ecd14464bbc2a177/dask_labextension/dashboardhandler.py#L250-L277 So it's not really a ton of code, more of a matter of figuring out how to do it in a suitably user-friendly way. Any thoughts?

mrocklin commented 5 years ago

So, I'm also curious about this. Looking at the primary use cases listed in the readme I see the following:

Allow access from frontend javascript (in classic notebook or JupyterLab extensions) to access web APIs of other processes running locally in a safe manner. This is used by the JupyterLab extension for dask.

I think I'm curious now about what if the to-be-proxied server is living within the same process as the server, for example as part of a server extension. I can imagine using this with Dask, or also with other dashboards. For context today I'm working with @ian-r-rose on trying to make it easier to create dashboards with Bokeh and expose them through JupyterLab. My original thought was that the Bokeh server should just live within the same process as the Jupyter server living on the same event loop, rather than being a wholly separate process.