Notes on using Lonboard in a remote (esp JupyterHub) environment

[kylebarron]

I was pairing with @wildintellect on debugging why Lonboard wasn't working in a JupyterHub environment and we came up with a couple notes.

In particular, it's very easy to get lost in environments. Generally a JupyterHub server has at least two Python environments. There's a server environment in which JupyterHub itself is installed and from which it's launched, plus one or more user client environments. As far as I can tell, a Jupyter widget needs to be installed in both environments, and presumably those versions must match.

The widget installed on the server environment is necessary to provide the JS bundle as part of the webpage the user loads, while the widget installed on the user environment is necessary so the Python model works.

• Are there ways to provide helpful errors when no JS code is found? Only a JS console error is produced. Is there some way to catch that and provide a more helpful error to the user?
• Is there a good way to validate that the frontend and backend versions of lonboard match?

Aside from lonboard-specific tasks, I wonder if there are tasks that JupyterHub could do that would simplify the loading of widgets so that they don't need to be separately pre-installed on the server. Additionally, ensuring versions match between the user's environment and the global server environment sounds messy.

I've noticed that Colab handles this problem quite well (I think it's with their googlecolab/colab-cdn-widget-manager project). In particular, the first time that a widget is attempted to be rendered, the JS code for that widget is dynamically added to the page. This means that

!pip install lonboard
from lonboard import viz
viz([])

will just work, even without restarting the kernel or reloading the browser tab, because the JS side doesn't need to exist already when the Python side is created.

cc @batpad

[kylebarron]

Unsurprisingly, there's been discussion on this before. See especially https://github.com/manzt/anywidget/issues/385#issuecomment-1821263446.

Note that the architecture of anywidget is that anywidget serves as a bridge between "fully bundled" widgets that need to be installed in full on the server and "dynamically imported" widgets. Anywidget itself loads the JS from the anywidget-based widget. Therefore only a compatible version of anywidget needs to be installed on the server. In our case, lonboard does not need to be installed on the server, only anywidget.

There are further questions about ensuring compatible versions of anywidget between frontend and server, but that's a more general question outside of lonboard's scope.

Solara also does some nice checks around this: https://github.com/manzt/anywidget/issues/385#issuecomment-1972693336

[paddymul]

FWIW In my own IPYwidget library buckaroo, I addressed some of this.
Users frequently pip installed the library without restarting the server. This causes a JS error with no python error. I added a check on the MTime of the python library, and the start time of the server.

https://github.com/paddymul/buckaroo/blob/main/buckaroo/widget_utils.py#L37-L55

I'm interested to hear about other approaches around this problem. It's a really poor experience for an initial user install. Maybe you could source the JS from unpackage when the user would encounter the problem.

[ymoisan]

discussion on this before. See especially https://github.com/manzt/anywidget/issues/385#issuecomment-1821263446.

says "... a virtual environment (Python venv, Conda environment or what have you) where package jupyterlab has been deployed — one deploys their widget package in the same environment as where Jupyter Lab lives" (bold mine).

When I was running for example your NYC building notebook from a Jupyter Lab instance launched from my jupyterlab conda env -- to which I had made my lonboard conda env visible as a kernel with the usual ipykernel dance -- I was having rendering issues e.g.

[1] m
Map(layers=[PolygonLayer(extruded=True, get_elevation=<pyarrow.lib.FloatArray object at 0x0000028B03972680>
[
…

And similar errors for ScatterPlot layers and viz([])

In order to fix this, I just installed jupyterlab directly in my lonboard env, launched JLab from that env and bingo.

Unless there's a "JS dance" to expose the JS of conda envs in kernels exported to jupyterlab, I think that means we need to duplicate jupyterlab installs for those conda envs where there is JS.

lonboard v. '0.9.3' in Jupyter Lab 4.2.1

[kylebarron]

I just installed jupyterlab directly in my lonboard env, launched JLab from that env and bingo.

FWIW this is what I usually do personally.

For other cases, all you need is to install anywidget in the JupyterLab environment, so that anywidget's JS code exists in JupyterLab when you start the page. Then you can install lonboard separately inside a user environment. You do have to ensure that the anywidget version on the backend (in JupyterLab) and anywidget in the user environment is semver compatible.