Pydeck (jupyter lab) does not work together with keplergl

[bergkvist]

I can't seem to get both keplergl and pydeck widgets to work in JupyterLab at the same time

keplergl alone works fine

pip install keplergl==0.2.1 && jupyter labextension install keplergl-jupyter@0.2.1

import keplergl
display(keplergl.KeplerGl()) # Displays map

pydeck alone works fine

pip install pydeck==0.4.1 && jupyter labextension install @deck.gl/jupyter-widget@8.2.7

import pydeck
display(pydeck.Deck(mapbox_key='...').show()) # Displays map

---

keplergl and pydeck at the same time does not work

pip install keplergl==0.2.1 && jupyter labextension install keplergl-jupyter@0.2.1
pip install pydeck==0.4.1 && jupyter labextension install @deck.gl/jupyter-widget@8.2.7

import pydeck
display(pydeck.Deck(mapbox_key='...').show()) # Error displaying widget: model not found

import keplergl
display(keplergl.KeplerGl()) # Displays map

---

Reversing installation order reverses the problem:

pip install pydeck==0.4.1 && jupyter labextension install @deck.gl/jupyter-widget@8.2.7
pip install keplergl==0.2.1 && jupyter labextension install keplergl-jupyter@0.2.1

import pydeck
display(pydeck.Deck(mapbox_key='...').show()) # Displays map

import keplergl
display(keplergl.KeplerGl()) # Error displaying widget: model not found

[kylebarron]

It's most likely a deck.gl versioning problem. There's no sandbox in the browser between widgets, and pydeck and keplergl aren't released together, so there's no easy way to be sure that the underlying JS part of the widget is using the same deck.gl version. Because there's no sandbox, the deck version from one will overwrite the deck provided with the other.

I think the necessary workaround is to use separate browser tabs or at least reload the page when switching between pydeck and keplergl.

[bergkvist]

Reloading the browser/using different tabs doesn't have any effect. To switch between keplergl and pydeck I actually need to uninstall/reinstall/rebuild the widgets. And this takes a long time (~10-15 minutes on my laptop).

An alternative would be to create 2 separate docker images and run both of them side-by-side (that way I don't need to wait 10-15 min every time I want to switch).

[bergkvist]

Since jupyterlab extensions are built using webpack, perhaps something like this could be used to avoid deck.gl version conflicts?

• https://stackoverflow.com/questions/49323299/can-webpack-bundle-different-versions-of-the-same-dependency
• https://stackoverflow.com/questions/26414587/how-to-install-multiple-versions-of-package-using-npm

// package.json
"dependencies": {
   "keplergl-deck.gl": "npm:deck.gl@8.2.5",
   "pydeck-deck.gl": "npm:deck.gl@8.2.7"
}

This syntax is available since npm v6.9.0 (Mar 6, 2019)

[ibgreen]

While I agree that deck.gl versions could be a source of problems, the error messages you are receiving seem more related to the jupyter widget API (jupyterWidgetRegistry, get_model, renderModel).

It could of course be some deck issue that prevents initialization but normally I would expect jupyter to initialize first.

@ajduberstein

[kylebarron]

FYI I tried this out and I see in the browser console:

Error: deck.gl - multiple versions detected: 8.2.7 vs 8.2.5
    at Object.push.WQvm (vendors~main.bb9e8da21be7567696e5.js:391674)
    at r (vendors~main.bb9e8da21be7567696e5.js:391657)
    at Object.push.WQvm (vendors~main.bb9e8da21be7567696e5.js:391674)
    at r (vendors~main.bb9e8da21be7567696e5.js:391657)
    at Module.push.WQvm (vendors~main.bb9e8da21be7567696e5.js:391657)
    at r (vendors~main.bb9e8da21be7567696e5.js:391657)
    at Object.<anonymous> (vendors~main.bb9e8da21be7567696e5.js:391674)
    at Object.push.WQvm (vendors~main.bb9e8da21be7567696e5.js:391674)
    at r (vendors~main.bb9e8da21be7567696e5.js:391657)
    at Object.push.WQvm (vendors~main.bb9e8da21be7567696e5.js:391657)

At first glance it looks like that's the cause of the widget model not found errors.

[bergkvist]

This might have been caused by the JupyterLab version (v1.2.5). I'm not 100% sure if this is the reason - but I suddenly got both widgets working at the same time in JupyterLab v2.2.5 now.

In case you are interested, the docker image where I got it working is available at bergkvist/kognilab:v1.1.4 (Docker Hub):

Dockerfile (warning: quite large)

```dockerfile FROM jupyter/scipy-notebook:6d42503c684f as builder ENV NODE_OPTIONS="--max-old-space-size=4096" RUN pip install --no-cache-dir jupyter-lsp python-language-server[all] dask_labextension ipympl &&\ jupyter serverextension enable dask_labextension RUN jupyter labextension install \ jupyterlab-plotly@4.9.0 \ plotlywidget@4.9.0 \ @jupyter-widgets/jupyterlab-manager && \ rm -rf /opt/conda/share/jupyter/lab/staging && \ fix-permissions $CONDA_DIR $HOME # Split into 2 steps in order to reduce peak memory consumption RUN jupyter labextension install \ @krassowski/jupyterlab-lsp \ dask-labextension \ @deck.gl/jupyter-widget@8.2.7 \ keplergl-jupyter@0.2.1 && \ rm -rf /opt/conda/share/jupyter/lab/staging && \ fix-permissions $CONDA_DIR $HOME # Compress 5 copy instructions into a single layer in the final image FROM scratch as labextensions COPY --from=builder /home/jovyan/.config /home/jovyan/.config COPY --from=builder /home/jovyan/.jupyter /home/jovyan/.jupyter COPY --from=builder /opt/conda/etc/jupyter /opt/conda/etc/jupyter COPY --from=builder /opt/conda/share/jupyter /opt/conda/share/jupyter COPY --from=builder /opt/conda/lib/python3.8/site-packages /opt/conda/lib/python3.8/site-packages FROM jupyter/scipy-notebook:6d42503c684f COPY --from=labextensions / / COPY --from=graphblas/lagraph:13June2020 / / USER root RUN fix-permissions $CONDA_DIR $HOME RUN ldconfig RUN apt-get update && apt-get install -yq --no-install-recommends \ libgdal-dev \ libsnappy-dev && \ apt-get clean && rm -rf /var/lib/apt/lists/* USER jovyan RUN pip install --no-cache-dir pygeos --no-binary pygeos pygdal==3.0.4.6 pyproj python-snappy fastparquet pyarrow && \ pip install --no-cache-dir keplergl==0.2.1 pydeck==0.4.1 plotly==4.9.0 matplotlib seaborn tqdm && \ pip install --no-cache-dir pandapower==2.3.1 scipy==1.5.2 numpy==1.19.1 dask memoization numba && \ pip install --no-cache-dir email-validator aiofiles python-dotenv livereload==2.6.2 fastapi uvicorn && \ pip install --no-cache-dir pytest==5.4.3 pytest-watch==4.2.0 pytest-testmon==1.0.2 pytest-html==2.1.1 taskipy==1.2.1 && \ pip install --no-cache-dir jedi==0.17.2 parso==0.7.1 coverage==5.2.1 line_profiler==3.0.2 memory_profiler==0.57.0 && \ pip install --no-cache-dir psycopg2-binary geoalchemy2 lxml xlrd requests humanize poetry ENV JUPYTER_ENABLE_LAB=yes ```

I'm also not getting any errors regarding conflicting deck.gl-versions in this Docker image.

[ajduberstein]

Thanks for filing the issue, @bergkvist–could we get the output of jupyter labextension list for the broken and working environments? This will help us fix this same problem for other users in the future.

[bergkvist]

@ajduberstein

Broken environment (jupyter labextension list):

JupyterLab v1.2.5
Known labextensions:
   app dir: /usr/share/jupyter/lab
        @deck.gl/jupyter-widget v8.2.7  enabled  OK
        @jupyter-widgets/jupyterlab-manager v1.1.0  enabled  OK
        jupyter-matplotlib v0.5.0  enabled  OK
        jupyterlab-plotly v4.8.0  enabled  OK
        keplergl-jupyter v0.2.1  enabled  OK
        plotlywidget v4.8.0  enabled  OK

Working environment (jupyter labextension list):

JupyterLab v2.2.5
Known labextensions:
   app dir: /opt/conda/share/jupyter/lab
        @bokeh/jupyter_bokeh v2.0.3  enabled  OK
        @deck.gl/jupyter-widget v8.2.7  enabled  OK
        @jupyter-widgets/jupyterlab-manager v2.0.0  enabled  OK
        @krassowski/jupyterlab-lsp v1.1.2  enabled  OK
        dask-labextension v3.0.0  enabled  OK
        jupyter-matplotlib v0.7.3  enabled  OK
        jupyterlab-plotly v4.9.0  enabled  OK
        keplergl-jupyter v0.2.1  enabled  OK
        plotlywidget v4.9.0  enabled  OK

Notice @jupyter-widgets/jupyterlab-manager, which has different major versions (and is a dependency of keplergl-jupyter and @deck.gl/jupyter-widget).