search

keplergl / kepler.gl

Kepler.gl is a powerful open source geospatial analysis tool for large-scale data sets.
http://kepler.gl
MIT License
10.45k stars 1.75k forks source link

[Bug][Jupyter Widget] Kepler will not display a map in docker if built non-interactively #1581

Open amerenda opened 3 years ago

amerenda commented 3 years ago

I have also opened an issue in the Jupyter Docker stacks repo: https://github.com/jupyter/docker-stacks/issues/1434

Describe the bug Kepler will not display a map if built in a docker container via a script. But will display the map if built via a terminal emulator. This behavior is consistent when built both ways on the same VM, with the same Dockerfile and same version of docker. Running jupyter lab build on the image built via a script will allow the map to load correctly.

To Reproduce Steps to reproduce the behavior:

  1. Create a docker image
  2. Build it headless (via script, ssh, or build automation tool, I'm using Google Cloud Build in my case, but I can reproduce with a script running locally or on a VM)
  3. Load Jupyter and create a Kepler map

Dockerfile

FROM jupyter/base-notebook:lab-3.1.6
RUN jupyter lab clean --all && \
    pip install --no-cache-dir --upgrade keplergl && \
    jupyter labextension install @jupyter-widgets/jupyterlab-manager keplergl-jupyter

Script to create

#!/usr/bin/env bash
DOCKER_PATH=/home/alexm
IMAGE_NAME=kepler
TAG=script

# Build the image
docker build $DOCKER_PATH \
  -t $IMAGE_NAME:$TAG

Map code

from keplergl import KeplerGl
import keplergl
map = KeplerGl()
map

Expected behavior Kepler loads a map

Screenshots https://imgur.com/XM5I3gs

Environment (please complete the following information):

Additional context If you run this command via bash/zsh from a terminal docker build $DOCKER_PATH -t $IMAGE_NAME:$TAG run the container, and load the kepler map, it will display correctly.

DOCKER_PATH=/home/alexm
IMAGE_NAME=kepler
TAG=terminal

docker build $DOCKER_PATH \
  -t $IMAGE_NAME:$TAG

When running the docker build via a script it fails.

To eliminate any environmental sources, I ran the build manually on the VM, confirmed it worked, and ran the script on the same VM with the same Dockerfile and confirmed it did not work.

I thought that maybe it was related to docker being called from a bash pipe and not a TTY? But I called it via tmux in a script, confirmed it was running in a TTY, and I had the same issue. 

#!/usr/bin/env bash
DOCKER_PATH=/home/alexm
IMAGE_NAME=kepler
TAG=script
tmux new-session -d -s docker-build "docker build $DOCKER_PATH -f  \
  -t $IMAGE_NAME:$TAG"

# Wait for the build to finish
while tmux ls | grep 'docker-build' > /dev/null
do
  sleep 1
done

I also thought maybe there was a rogue environment variable being set, so I compared the output of env on both a script and via the terminal. The differences are:

Terminal

TERM=screen-256color
SHLVL=1

Script

SHLVL=2

I also run the build via a terminal inside a tmux session where SHLVL=2 and I set TERM=sceen-256color and confirmed the map loaded.

Terminal env

SHELL=/bin/bash
SSH_AUTH_SOCK=/tmp/ssh-YNMxCpXaV4/agent.27474
PWD=/home/alexm
LOGNAME=alexm
XDG_SESSION_TYPE=tty
HOME=/home/alexm
LANG=C.UTF-8
LS_COLORS=rs=0:di=01;34:ln=01;36:mh=00:pi=40;33:so=01;35:do=01;35:bd=40;33;01:cd=40;33;01:or=40;31;01:mi=00:su=37;41:sg=30;43:ca=30;41:tw=30;42:ow=34;42:st=37;44:ex=01;32:*.tar=01;31:*.tgz=01;31:*.arc=01;31:*.arj=01;31:*.taz=01;31:*.lha=01;31:*.lz4=01;31:*.lzh=01;31:*.lzma=01;31:*.tlz=01;31:*.txz=01;31:*.tzo=01;31:*.t7z=01;31:*.zip=01;31:*.z=01;31:*.dz=01;31:*.gz=01;31:*.lrz=01;31:*.lz=01;31:*.lzo=01;31:*.xz=01;31:*.zst=01;31:*.tzst=01;31:*.bz2=01;31:*.bz=01;31:*.tbz=01;31:*.tbz2=01;31:*.tz=01;31:*.deb=01;31:*.rpm=01;31:*.jar=01;31:*.war=01;31:*.ear=01;31:*.sar=01;31:*.rar=01;31:*.alz=01;31:*.ace=01;31:*.zoo=01;31:*.cpio=01;31:*.7z=01;31:*.rz=01;31:*.cab=01;31:*.wim=01;31:*.swm=01;31:*.dwm=01;31:*.esd=01;31:*.jpg=01;35:*.jpeg=01;35:*.mjpg=01;35:*.mjpeg=01;35:*.gif=01;35:*.bmp=01;35:*.pbm=01;35:*.pgm=01;35:*.ppm=01;35:*.tga=01;35:*.xbm=01;35:*.xpm=01;35:*.tif=01;35:*.tiff=01;35:*.png=01;35:*.svg=01;35:*.svgz=01;35:*.mng=01;35:*.pcx=01;35:*.mov=01;35:*.mpg=01;35:*.mpeg=01;35:*.m2v=01;35:*.mkv=01;35:*.webm=01;35:*.ogm=01;35:*.mp4=01;35:*.m4v=01;35:*.mp4v=01;35:*.vob=01;35:*.qt=01;35:*.nuv=01;35:*.wmv=01;35:*.asf=01;35:*.rm=01;35:*.rmvb=01;35:*.flc=01;35:*.avi=01;35:*.fli=01;35:*.flv=01;35:*.gl=01;35:*.dl=01;35:*.xcf=01;35:*.xwd=01;35:*.yuv=01;35:*.cgm=01;35:*.emf=01;35:*.ogv=01;35:*.ogx=01;35:*.aac=00;36:*.au=00;36:*.flac=00;36:*.m4a=00;36:*.mid=00;36:*.midi=00;36:*.mka=00;36:*.mp3=00;36:*.mpc=00;36:*.ogg=00;36:*.ra=00;36:*.wav=00;36:*.oga=00;36:*.opus=00;36:*.spx=00;36:*.xspf=00;36:
SSH_CONNECTION=10.80.0.2 45994 10.80.79.202 22
XDG_SESSION_CLASS=user
TERM=screen-256color
USER=alexm
SHLVL=1
XDG_SESSION_ID=61
XDG_RUNTIME_DIR=/run/user/1005
SSH_CLIENT=10.80.0.2 45994 22
PATH=/usr/local/bin:/usr/bin:/bin:/usr/local/games:/usr/games
MAIL=/var/mail/alexm
SSH_TTY=/dev/pts/0
_=/usr/bin/env
OLDPWD=/home/alexm/galileo

Script env

SHELL=/bin/bash
SSH_AUTH_SOCK=/tmp/ssh-YNMxCpXaV4/agent.27474
PWD=/home/alexm
LOGNAME=alexm
XDG_SESSION_TYPE=tty
_=/usr/bin/env
HOME=/home/alexm
LANG=C.UTF-8
LS_COLORS=rs=0:di=01;34:ln=01;36:mh=00:pi=40;33:so=01;35:do=01;35:bd=40;33;01:cd=40;33;01:or=40;31;01:mi=00:su=37;41:sg=30;43:ca=30;41:tw=30;42:ow=34;42:st=37;44:ex=01;32:*.tar=01;31:*.tgz=01;31:*.arc=01;31:*.arj=01;31:*.taz=01;31:*.lha=01;31:*.lz4=01;31:*.lzh=01;31:*.lzma=01;31:*.tlz=01;31:*.txz=01;31:*.tzo=01;31:*.t7z=01;31:*.zip=01;31:*.z=01;31:*.dz=01;31:*.gz=01;31:*.lrz=01;31:*.lz=01;31:*.lzo=01;31:*.xz=01;31:*.zst=01;31:*.tzst=01;31:*.bz2=01;31:*.bz=01;31:*.tbz=01;31:*.tbz2=01;31:*.tz=01;31:*.deb=01;31:*.rpm=01;31:*.jar=01;31:*.war=01;31:*.ear=01;31:*.sar=01;31:*.rar=01;31:*.alz=01;31:*.ace=01;31:*.zoo=01;31:*.cpio=01;31:*.7z=01;31:*.rz=01;31:*.cab=01;31:*.wim=01;31:*.swm=01;31:*.dwm=01;31:*.esd=01;31:*.jpg=01;35:*.jpeg=01;35:*.mjpg=01;35:*.mjpeg=01;35:*.gif=01;35:*.bmp=01;35:*.pbm=01;35:*.pgm=01;35:*.ppm=01;35:*.tga=01;35:*.xbm=01;35:*.xpm=01;35:*.tif=01;35:*.tiff=01;35:*.png=01;35:*.svg=01;35:*.svgz=01;35:*.mng=01;35:*.pcx=01;35:*.mov=01;35:*.mpg=01;35:*.mpeg=01;35:*.m2v=01;35:*.mkv=01;35:*.webm=01;35:*.ogm=01;35:*.mp4=01;35:*.m4v=01;35:*.mp4v=01;35:*.vob=01;35:*.qt=01;35:*.nuv=01;35:*.wmv=01;35:*.asf=01;35:*.rm=01;35:*.rmvb=01;35:*.flc=01;35:*.avi=01;35:*.fli=01;35:*.flv=01;35:*.gl=01;35:*.dl=01;35:*.xcf=01;35:*.xwd=01;35:*.yuv=01;35:*.cgm=01;35:*.emf=01;35:*.ogv=01;35:*.ogx=01;35:*.aac=00;36:*.au=00;36:*.flac=00;36:*.m4a=00;36:*.mid=00;36:*.midi=00;36:*.mka=00;36:*.mp3=00;36:*.mpc=00;36:*.ogg=00;36:*.ra=00;36:*.wav=00;36:*.oga=00;36:*.opus=00;36:*.spx=00;36:*.xspf=00;36:
SSH_CONNECTION=10.80.0.2 45994 10.80.79.202 22
XDG_SESSION_CLASS=user
TERM=screen-256color
USER=alexm
SHLVL=2
XDG_SESSION_ID=61
XDG_RUNTIME_DIR=/run/user/1005
SSH_CLIENT=10.80.0.2 45994 22
PATH=/usr/local/bin:/usr/bin:/bin:/usr/local/games:/usr/games
MAIL=/var/mail/alexm
SSH_TTY=/dev/pts/0
OLDPWD=/home/alexm/galileo

I get the following console errors if I use the Chrome dev tools when built via a script.

index.es6.js:282 Plugin 'keplergl-jupyter' failed to activate.
(anonymous) @ index.es6.js:282
index.es6.js:283 Error: No provider for: jupyter.extensions.jupyterWidgetRegistry.
    at W.e.resolveRequiredService (index.es6.js:197)
    at index.es6.js:155
    at Array.map (<anonymous>)
    at W.e.activatePlugin (index.es6.js:155)
    at index.es6.js:281
    at Array.map (<anonymous>)
    at W.e.start (index.es6.js:280)
    at I (index.out.js:625)
(anonymous) @ index.es6.js:283
manager-base.js:274 Could not instantiate widget
(anonymous) @ manager-base.js:274
utils.js:119 Error: Could not create a model.
    at n (utils.js:119)
    at async _handleCommOpen (523.fa256ee012d38a89b65a.js:1)
    at async b._handleCommOpen (default.js:1020)
    at async b._handleMessage (default.js:1188)
n @ utils.js:119
default.js:1027 Exception opening new comm
_handleCommOpen @ default.js:1027
523.fa256ee012d38a89b65a.js:1 Error: Module keplergl-jupyter, semver range ^0.3.0 is not registered as a widget module
    at x.loadClass (523.fa256ee012d38a89b65a.js:1)
    at x.<anonymous> (manager-base.js:264)
    at l (manager-base.js:45)
    at Object.next (manager-base.js:26)
    at manager-base.js:20
    at new Promise (<anonymous>)
    at A (manager-base.js:16)
    at x.e._make_model (manager-base.js:258)
    at x.<anonymous> (manager-base.js:247)
    at l (manager-base.js:45)
renderModel @ 523.fa256ee012d38a89b65a.js:1
2523.fa256ee012d38a89b65a.js:1 Uncaught (in promise) Error: Module keplergl-jupyter, semver range ^0.3.0 is not registered as a widget module
    at x.loadClass (523.fa256ee012d38a89b65a.js:1)
    at x.<anonymous> (manager-base.js:264)
    at l (manager-base.js:45)
    at Object.next (manager-base.js:26)
    at manager-base.js:20
    at new Promise (<anonymous>)
    at A (manager-base.js:16)
    at x.e._make_model (manager-base.js:258)
    at x.<anonymous> (manager-base.js:247)
    at l (manager-base.js:45)

I've never seen anything like this issue and I'm not sure how to proceed. I don't know if it is a docker issue, a kepler issue, or an NPM issue? If anyone has any insights, it would be greatly appreciated. For now, I cannot get Kepler to run on Jupyter unless I build my container manually.

kopp commented 3 years ago

This sound really strange. Just to confirm: If you write docker build -t foo . in your shell and produce a docker image tagged foo it loads the map just fine, but if you have a script file with the content docker build -t bar ., run bash <yourscriptfile> and run the docker image tagged bar it fails to render the map? What makes me wonder most: If you re-run building the same image (either from script or from the shell) docker should use caching (i.e. not re-build the docker layers) -- so if you build using docker build -t foo . on the shell once and then bash <yourscriptfile> I expect docker to simply re-tag the same image -- does it work then?

I also stumble over the same error you reported, but I think that it is related to the wrong version of the javascript code being "encoded" in the python wrapping code.

In my case, I get the following in a terminal in the JupyterLab (where the map fails to work): jupyter labextension list reports version 0.3.0 for keplergl, while pip show keplergl reports version 0.3.1. So I suspect that the error you (and I) see is simply due to this version mismatch...

I hope that this will be solved soonish once https://github.com/keplergl/kepler.gl/commit/5b442c5dda5a423261f0f663bd956f31016092b7 is published to pypi.org.