Run jupyter-dash with Jupyterlab2.x + JupyterHub + Kubernetes

[rdelubi]

Regarding #2, not sure which server proxy it is referring to. Can you point me to the link to the documentation?

Hi @rdelubi, This issue is specifically concerning Colab, because Colab isn't a standard Jupyter front-end. For regular JupyterHub, it should be enough to:

1. Make sure the jupyterlab-dash extension is built (https://github.com/plotly/jupyter-dash#jupyterlab-support)
2. Make sure jupyter_server_proxy is installed in the environment running the Jupyter server.
3. Call JupyterDash.infer_jupyter_proxy_config() at the top of the notebook. This uses the JupyterLab extension to detect the URL that JupyterLab is being accessed at (This is what server_url is), passes that info back to the Python library for use in configuring Dash.

If that doesn't work for you, could you open a new issue? Thanks!

Originally posted by @jonmmease in https://github.com/plotly/jupyter-dash/issues/10#issuecomment-633693906

[jonmmease]

Hi @rdelubi, Jupyer server proxy is this python package (https://github.com/jupyterhub/jupyter-server-proxy).

[rdelubi]

@jonmmease Tried the followings:

The output from infer shows none.

My notebook environment:

sh-4.2$ jupyter labextension list
JupyterLab v2.1.2
Known labextensions:
   app dir: /opt/app-root/share/jupyter/lab
        @jupyterlab/debugger v0.3.0-alpha.1  enabled  OK
        @jupyterlab/git v0.20.0  enabled  OK
        @jupyterlab/server-proxy v2.1.0  enabled  OK
        jupyterlab-dash v0.2.0  enabled  OK
        jupyterlab-sql v0.3.3  enabled   X
        nbdime-jupyterlab v2.0.0  enabled  OK

   The following extension are outdated:
        jupyterlab-sql

   Consider running "jupyter labextension update --all" to check for updates.

sh-4.2$

sh-4.2$ pip list |grep jupyter
jupyter-client         6.1.3
jupyter-core           4.6.3
jupyter-dash           0.2.1.post1
jupyter-kernel-gateway 2.4.0
jupyter-server-proxy   1.5.0
jupyterhub             1.0.0
jupyterlab             2.1.2
jupyterlab-dash        0.1.0a3
jupyterlab-git         0.20.0
jupyterlab-latex       2.0.0
jupyterlab-server      1.1.5
jupyterlab-sql         0.3.3
WARNING: You are using pip version 20.1; however, version 20.1.1 is available.
You should consider upgrading via the '/opt/app-root/bin/python3 -m pip install --upgrade pip' command.
sh-4.2$

Any thoughts what I may be missing?

[jonmmease]

That config looks alright. If infer_jupyter_proxy_config returns without an exception then that means the JupyterLab extension is connecting. Here are a few more things to look at:

Could you post the result of:

from jupyter_dash.comms import _jupyter_config
_jupyter_config

run this in a cell after JupyterDash.infer_jupyter_proxy_config()

---

What URL is printed out when you run:

app.run_server(mode="external")

---

When you open that URL. What do you see? is it a 404 error, does it get stuck on Loading..., etc.?

---

What is the full URL of JupyterLab where you are running it?

---

What is the value of JupyterDash()._server_proxy?

---

Can you successfully import jupyter_server_proxy from the notebook?

import jupyter_server_proxy

This info should give us some clues into what's going on.

Also, I don't think it would cause a problem, but I see that you have old jupyterlab-dash Python package installed. It might be worth trying to uninstall that package just in case.

[rdelubi]

That config looks alright. If infer_jupyter_proxy_config returns without an exception then that means the JupyterLab extension is connecting. Here are a few more things to look at:

Could you post the result of:

from jupyter_dash.comms import _jupyter_config
_jupyter_config

run this in a cell after JupyterDash.infer_jupyter_proxy_config()

Execution output of cell above erver_url': 'https://jupyterhub-<k8s namespace + domain name>', 'base_subpath': '/user//', 'frontend': 'jupyterlab'}

What URL is printed out when you run:

app.run_server(mode="external")

So I run a cell as

app = JupyterDash(name, server_url="https://jupyterhub-<k8s namespace + domain name>/user/xxxxxx/") app.run_server(mode="external")

The output indicates port already in use

---------------------------------------------------------------------------
OSError                                   Traceback (most recent call last)
<ipython-input-6-3e00f1c36cff> in <module>
      1 app = JupyterDash(__name__, server_url="https://jupyterhub-dev-cl1-tsa0-gam-cloud-devops-tools.apps.c1-ocp-dc1.devfg.rbc.com/user/845984392/")
----> 2 app.run_server(mode="external")

/opt/app-root/lib/python3.6/site-packages/jupyter_dash/jupyter_app.py in run_server(self, mode, width, height, inline_exceptions, **kwargs)
    292                 )
    293 
--> 294         wait_for_app()
    295 
    296         if mode == 'inline':

/opt/app-root/lib/python3.6/site-packages/retrying.py in wrapped_f(*args, **kw)
     47             @six.wraps(f)
     48             def wrapped_f(*args, **kw):
---> 49                 return Retrying(*dargs, **dkw).call(f, *args, **kw)
     50 
     51             return wrapped_f

/opt/app-root/lib/python3.6/site-packages/retrying.py in call(self, fn, *args, **kwargs)
    210                 if not self._wrap_exception and attempt.has_exception:
    211                     # get() on an attempt with an exception should cause it to be raised, but raise just in case
--> 212                     raise attempt.get()
    213                 else:
    214                     raise RetryError(attempt)

/opt/app-root/lib/python3.6/site-packages/retrying.py in get(self, wrap_exception)
    245                 raise RetryError(self)
    246             else:
--> 247                 six.reraise(self.value[0], self.value[1], self.value[2])
    248         else:
    249             return self.value

/opt/app-root/lib/python3.6/site-packages/six.py in reraise(tp, value, tb)
    701             if value.__traceback__ is not tb:
    702                 raise value.with_traceback(tb)
--> 703             raise value
    704         finally:
    705             value = None

/opt/app-root/lib/python3.6/site-packages/retrying.py in call(self, fn, *args, **kwargs)
    198      

When you open that URL. What do you see? is it a 404 error, does it get stuck on Loading..., etc.?

If I open another tab and go to URL at https://jupyterhub-<k8s namespace + domain name> I get the jupyterlab workspace.

What is the full URL of JupyterLab where you are running it?

Full url shown in browser is: https://jupyterhub-<k8s namespace + domain name>/user/xxxxxx/lab?

What is the value of JupyterDash()._server_proxy?

Value is "True"

Can you successfully import jupyter_server_proxy from the notebook?

import jupyter_server_proxy

Yes. Import has no error.

This info should give us some clues into what's going on.

Also, I don't think it would cause a problem, but I see that you have old jupyterlab-dash Python package installed. It might be worth trying to uninstall that package just in case.

Uninstall it. Didn't seem to make difference though.

[jonmmease]

Ok, well let's start with the port already in use part: Try calling run_server with the port argument to pass it a port other than the default 8050. You can just pick another value like 10213 and see if the error goes away, or you can programmatically search for an open port with this function:

def find_open_port():
    s = socket.socket()
    s.setsockopt(socket.SOL_SOCKET, socket.SO_REUSEADDR, 1)
    s.bind(("", 0))
    _, port = s.getsockname()
    s.close()

    return port

Call it with something like this:

port = find_open_port()
print("port: {port}".format(port=port))
app.run_server(mode="external", port=port)

I would expect it to print out a URL like this: https://jupyterhub-<k8s namespace + domain name>/user/proxy/{port}/

[rdelubi]

Tried the suggestion but output still showed port already in use. Please see screen snapshot below.

Executed again and output had a new port number available but then still said port already in use.

[jonmmease]

One last idea, try setting host to "0.0.0.0", "localhost", and the result of socket.getfqdn() in run_server:

app.run_server(mode="external", port=port, host="0.0.0.0")

or

app.run_server(mode="external", port=port, host="localhost")

or

import socket
host = socket.getfqdn();
app.run_server(mode="external", port=port, host=host)

Do any of these behave differently?

[rdelubi]

Tried all three variations. All had same output address/port already in use.

[gioxc88]

same here

[jonmmease]

@gioxc88 are you also running JupyterHub with Kubernetes?

[gioxc88]

@gioxc88 are you also running JupyterHub with Kubernetes?

No sorry I should have specified. I am just running it in jupyter lab

[stefaneidelloth]

I tried to use jupyter-dash with JupyterLab and JupyterHub inside a Docker container. The command JupyterDash.infer_jupyter_proxy_config() does not return. Do I need to map an extra port for jupyter-dash? Also see https://github.com/plotly/jupyter-dash/issues/16

[JarnoRFB]

@stefaneidelloth I had the same problem, meaning that JupyterDash.infer_jupyter_proxy_config() seemingly got stuck with a star. But then I found that I could simply execute the next cells and everything worked as expected. Very confusing though.

[GuySerfaty]

I see it also on lab 3.X

[umutkaya-ugent]

I can confirm that the following docker image can run jupyter dash on z2jh with kubernetes

FROM jupyter/minimal-notebook:2022-04-25

# install additional package
RUN pip install --no-cache-dir jupyter-server-proxy
RUN pip install --no-cache-dir jupyter-dash
RUN jupyter lab build

But I had to use a few work arounds (don't know if all of them are needed):

# %%
from jupyter_dash import JupyterDash as Dash
from jupyter_dash import JupyterDash

# %%
from jupyter_dash.comms import _send_jupyter_config_comm_request
_send_jupyter_config_comm_request()

# %%
JupyterDash.infer_jupyter_proxy_config()

# %%
import dash
from dash import dcc
from dash import html

# ... create the Jupyter dash app see: https://github.com/plotly/jupyter-dash/blob/master/notebooks/getting_started.ipynb

# %%
import jupyter_server_proxy

# %%
app.run_server(mode="jupyterlab")
# or
# app.run_server(mode="external", port=8053) # change the port accordingly

[KSeangTan]

Sorry for replying the old topic, but i would like to post my solution for others since have been working on this for few days

Hi, I found that it's server-proxy instead of proxy in my case so the link is https://jupyterhub-server/user/{username}/server-proxy/{port}

and we need to instantiate our dash app this way app = JupyterDash(__name__, requests_pathname_prefix=f"/user/{username}/server-proxy/{port}/", serve_locally=False)