search

jupyter / docker-stacks

Ready-to-run Docker images containing Jupyter applications
https://jupyter-docker-stacks.readthedocs.io
Other
7.99k stars 2.99k forks source link

[BUG] Jupyter lab build works inconsistently when the docker image is built via a script and built via a terminal. #1434

Closed amerenda closed 2 years ago

amerenda commented 3 years ago

This is a copy of: https://github.com/keplergl/kepler.gl/issues/1581 as I'm not sure if this bug has to do with kepler, or jupyter lab.

Describe the bug A kepler map built via the kepler widget will not display a map if built in a docker container via a script. But will display the map if built via a terminal emulator. This behavior is consistent when built both ways on the same VM, with the same Dockerfile and same version of docker. Running jupyter lab build on the image built via a script will allow the map to load correctly.

What docker image you are using? jupyter/base-notebook:lab-3.1.6 I have also tried multiple versions of jupyter/base-notebook jupyter/base-notebook:lab-3.1.4 jupyter/base-notebook:lab-3.0.16 jupyter/base-notebook:lab-3.0.15 jupyter/base-notebook:lab-3.0.14 jupyter/base-notebook:lab-2.2.9 And jupyter/minimal-notebook

To Reproduce Steps to reproduce the behavior:

  1. Create a docker image
  2. Build it headless (via script, ssh, or build automation tool, I'm using Google Cloud Build in my case, but I can reproduce with a script running locally or on a VM)
  3. Load Jupyter and create a Kepler map

Dockerfile

FROM jupyter/base-notebook:lab-3.1.6
RUN jupyter lab clean --all && \
    pip install --no-cache-dir --upgrade keplergl && \
    jupyter labextension install @jupyter-widgets/jupyterlab-manager keplergl-jupyter

Script to create

#!/usr/bin/env bash
DOCKER_PATH=/home/alexm
IMAGE_NAME=kepler
TAG=script

# Build the image
docker build $DOCKER_PATH \
  -t $IMAGE_NAME:$TAG

What complete docker command do you run to launch the container (omitting sensitive values)?

IMAGE_NAME=kepler
IMAGE_TAG=script
docker run -it -p 8080 $IMAGE_NAME:$IMAGE_TAG 'jupyter lab --port 8080

What steps do you take once the container is running to reproduce the issue? Open a python3 notebook Add 

from keplergl import KeplerGl
import keplergl
map = KeplerGl()
map

Expected behavior Kepler loads a map

Screenshots https://imgur.com/XM5I3gs

Environment (please complete the following information):

Additional context If you run this command via bash/zsh from a terminal docker build $DOCKER_PATH -t $IMAGE_NAME:$TAG run the container, and load the kepler map, it will display correctly without running jupyter lab build

DOCKER_PATH=/home/alexm
IMAGE_NAME=kepler
TAG=terminal

docker build $DOCKER_PATH \
  -t $IMAGE_NAME:$TAG

When running the docker build via a script, the build succeeds but the map will not work until jupyter lab build is run on the container.

To eliminate any environmental sources, I ran the build manually on the VM, confirmed it worked, and ran the script on the same VM with the same Dockerfile and confirmed it did not work.

I thought that maybe it was related to docker being called from a bash pipe and not a TTY? But I called it via tmux in a script, confirmed it was running in a TTY, and I had the same issue. 

#!/usr/bin/env bash
DOCKER_PATH=/home/alexm
IMAGE_NAME=kepler
TAG=script
tmux new-session -d -s docker-build "docker build $DOCKER_PATH -f  \
  -t $IMAGE_NAME:$TAG"

# Wait for the build to finish
while tmux ls | grep 'docker-build' > /dev/null
do
  sleep 1
done

I also thought maybe there was a rogue environment variable being set, so I compared the output of env on both a script and via the terminal. The differences are:

Terminal

SHLVL=1

Script

SHLVL=2

Tmux run from a script

SHLVL=2
TERM=screen

I also run the build via a terminal inside a tmux session where SHLVL=2 and I set TERM=screen-256color and confirmed the map loaded.

Terminal env

SHELL=/bin/bash
SSH_AUTH_SOCK=/tmp/ssh-YNMxCpXaV4/agent.27474
PWD=/home/alexm
LOGNAME=alexm
XDG_SESSION_TYPE=tty
HOME=/home/alexm
LANG=C.UTF-8
LS_COLORS=rs=0:di=01;34:ln=01;36:mh=00:pi=40;33:so=01;35:do=01;35:bd=40;33;01:cd=40;33;01:or=40;31;01:mi=00:su=37;41:sg=30;43:ca=30;41:tw=30;42:ow=34;42:st=37;44:ex=01;32:*.tar=01;31:*.tgz=01;31:*.arc=01;31:*.arj=01;31:*.taz=01;31:*.lha=01;31:*.lz4=01;31:*.lzh=01;31:*.lzma=01;31:*.tlz=01;31:*.txz=01;31:*.tzo=01;31:*.t7z=01;31:*.zip=01;31:*.z=01;31:*.dz=01;31:*.gz=01;31:*.lrz=01;31:*.lz=01;31:*.lzo=01;31:*.xz=01;31:*.zst=01;31:*.tzst=01;31:*.bz2=01;31:*.bz=01;31:*.tbz=01;31:*.tbz2=01;31:*.tz=01;31:*.deb=01;31:*.rpm=01;31:*.jar=01;31:*.war=01;31:*.ear=01;31:*.sar=01;31:*.rar=01;31:*.alz=01;31:*.ace=01;31:*.zoo=01;31:*.cpio=01;31:*.7z=01;31:*.rz=01;31:*.cab=01;31:*.wim=01;31:*.swm=01;31:*.dwm=01;31:*.esd=01;31:*.jpg=01;35:*.jpeg=01;35:*.mjpg=01;35:*.mjpeg=01;35:*.gif=01;35:*.bmp=01;35:*.pbm=01;35:*.pgm=01;35:*.ppm=01;35:*.tga=01;35:*.xbm=01;35:*.xpm=01;35:*.tif=01;35:*.tiff=01;35:*.png=01;35:*.svg=01;35:*.svgz=01;35:*.mng=01;35:*.pcx=01;35:*.mov=01;35:*.mpg=01;35:*.mpeg=01;35:*.m2v=01;35:*.mkv=01;35:*.webm=01;35:*.ogm=01;35:*.mp4=01;35:*.m4v=01;35:*.mp4v=01;35:*.vob=01;35:*.qt=01;35:*.nuv=01;35:*.wmv=01;35:*.asf=01;35:*.rm=01;35:*.rmvb=01;35:*.flc=01;35:*.avi=01;35:*.fli=01;35:*.flv=01;35:*.gl=01;35:*.dl=01;35:*.xcf=01;35:*.xwd=01;35:*.yuv=01;35:*.cgm=01;35:*.emf=01;35:*.ogv=01;35:*.ogx=01;35:*.aac=00;36:*.au=00;36:*.flac=00;36:*.m4a=00;36:*.mid=00;36:*.midi=00;36:*.mka=00;36:*.mp3=00;36:*.mpc=00;36:*.ogg=00;36:*.ra=00;36:*.wav=00;36:*.oga=00;36:*.opus=00;36:*.spx=00;36:*.xspf=00;36:
SSH_CONNECTION=10.80.0.2 45994 10.80.79.202 22
XDG_SESSION_CLASS=user
TERM=screen-256color
USER=alexm
SHLVL=1
XDG_SESSION_ID=61
XDG_RUNTIME_DIR=/run/user/1005
SSH_CLIENT=10.80.0.2 45994 22
PATH=/usr/local/bin:/usr/bin:/bin:/usr/local/games:/usr/games
MAIL=/var/mail/alexm
SSH_TTY=/dev/pts/0
_=/usr/bin/env
OLDPWD=/home/alexm/galileo

Script env

SHELL=/bin/bash
SSH_AUTH_SOCK=/tmp/ssh-YNMxCpXaV4/agent.27474
PWD=/home/alexm
LOGNAME=alexm
XDG_SESSION_TYPE=tty
_=/usr/bin/env
HOME=/home/alexm
LANG=C.UTF-8
LS_COLORS=rs=0:di=01;34:ln=01;36:mh=00:pi=40;33:so=01;35:do=01;35:bd=40;33;01:cd=40;33;01:or=40;31;01:mi=00:su=37;41:sg=30;43:ca=30;41:tw=30;42:ow=34;42:st=37;44:ex=01;32:*.tar=01;31:*.tgz=01;31:*.arc=01;31:*.arj=01;31:*.taz=01;31:*.lha=01;31:*.lz4=01;31:*.lzh=01;31:*.lzma=01;31:*.tlz=01;31:*.txz=01;31:*.tzo=01;31:*.t7z=01;31:*.zip=01;31:*.z=01;31:*.dz=01;31:*.gz=01;31:*.lrz=01;31:*.lz=01;31:*.lzo=01;31:*.xz=01;31:*.zst=01;31:*.tzst=01;31:*.bz2=01;31:*.bz=01;31:*.tbz=01;31:*.tbz2=01;31:*.tz=01;31:*.deb=01;31:*.rpm=01;31:*.jar=01;31:*.war=01;31:*.ear=01;31:*.sar=01;31:*.rar=01;31:*.alz=01;31:*.ace=01;31:*.zoo=01;31:*.cpio=01;31:*.7z=01;31:*.rz=01;31:*.cab=01;31:*.wim=01;31:*.swm=01;31:*.dwm=01;31:*.esd=01;31:*.jpg=01;35:*.jpeg=01;35:*.mjpg=01;35:*.mjpeg=01;35:*.gif=01;35:*.bmp=01;35:*.pbm=01;35:*.pgm=01;35:*.ppm=01;35:*.tga=01;35:*.xbm=01;35:*.xpm=01;35:*.tif=01;35:*.tiff=01;35:*.png=01;35:*.svg=01;35:*.svgz=01;35:*.mng=01;35:*.pcx=01;35:*.mov=01;35:*.mpg=01;35:*.mpeg=01;35:*.m2v=01;35:*.mkv=01;35:*.webm=01;35:*.ogm=01;35:*.mp4=01;35:*.m4v=01;35:*.mp4v=01;35:*.vob=01;35:*.qt=01;35:*.nuv=01;35:*.wmv=01;35:*.asf=01;35:*.rm=01;35:*.rmvb=01;35:*.flc=01;35:*.avi=01;35:*.fli=01;35:*.flv=01;35:*.gl=01;35:*.dl=01;35:*.xcf=01;35:*.xwd=01;35:*.yuv=01;35:*.cgm=01;35:*.emf=01;35:*.ogv=01;35:*.ogx=01;35:*.aac=00;36:*.au=00;36:*.flac=00;36:*.m4a=00;36:*.mid=00;36:*.midi=00;36:*.mka=00;36:*.mp3=00;36:*.mpc=00;36:*.ogg=00;36:*.ra=00;36:*.wav=00;36:*.oga=00;36:*.opus=00;36:*.spx=00;36:*.xspf=00;36:
SSH_CONNECTION=10.80.0.2 45994 10.80.79.202 22
XDG_SESSION_CLASS=user
TERM=screen-256color
USER=alexm
SHLVL=2
XDG_SESSION_ID=61
XDG_RUNTIME_DIR=/run/user/1005
SSH_CLIENT=10.80.0.2 45994 22
PATH=/usr/local/bin:/usr/bin:/bin:/usr/local/games:/usr/games
MAIL=/var/mail/alexm
SSH_TTY=/dev/pts/0
OLDPWD=/home/alexm/galileo

I get the following console errors if I use the Chrome dev tools when built via a script.

index.es6.js:282 Plugin 'keplergl-jupyter' failed to activate.
(anonymous) @ index.es6.js:282
index.es6.js:283 Error: No provider for: jupyter.extensions.jupyterWidgetRegistry.
    at W.e.resolveRequiredService (index.es6.js:197)
    at index.es6.js:155
    at Array.map (<anonymous>)
    at W.e.activatePlugin (index.es6.js:155)
    at index.es6.js:281
    at Array.map (<anonymous>)
    at W.e.start (index.es6.js:280)
    at I (index.out.js:625)
(anonymous) @ index.es6.js:283
manager-base.js:274 Could not instantiate widget
(anonymous) @ manager-base.js:274
utils.js:119 Error: Could not create a model.
    at n (utils.js:119)
    at async _handleCommOpen (523.fa256ee012d38a89b65a.js:1)
    at async b._handleCommOpen (default.js:1020)
    at async b._handleMessage (default.js:1188)
n @ utils.js:119
default.js:1027 Exception opening new comm
_handleCommOpen @ default.js:1027
523.fa256ee012d38a89b65a.js:1 Error: Module keplergl-jupyter, semver range ^0.3.0 is not registered as a widget module
    at x.loadClass (523.fa256ee012d38a89b65a.js:1)
    at x.<anonymous> (manager-base.js:264)
    at l (manager-base.js:45)
    at Object.next (manager-base.js:26)
    at manager-base.js:20
    at new Promise (<anonymous>)
    at A (manager-base.js:16)
    at x.e._make_model (manager-base.js:258)
    at x.<anonymous> (manager-base.js:247)
    at l (manager-base.js:45)
renderModel @ 523.fa256ee012d38a89b65a.js:1
2523.fa256ee012d38a89b65a.js:1 Uncaught (in promise) Error: Module keplergl-jupyter, semver range ^0.3.0 is not registered as a widget module
    at x.loadClass (523.fa256ee012d38a89b65a.js:1)
    at x.<anonymous> (manager-base.js:264)
    at l (manager-base.js:45)
    at Object.next (manager-base.js:26)
    at manager-base.js:20
    at new Promise (<anonymous>)
    at A (manager-base.js:16)
    at x.e._make_model (manager-base.js:258)
    at x.<anonymous> (manager-base.js:247)
    at l (manager-base.js:45)

JupyterLab logs:

[I 2021-08-20 13:50:56.937 ServerApp] jupyterlab | extension was successfully linked.
[W 2021-08-20 13:50:56.940 NotebookApp] 'ip' has moved from NotebookApp to ServerApp. This config will be passed to ServerApp. Be sure to update your config before our next release.
[W 2021-08-20 13:50:56.940 NotebookApp] 'port' has moved from NotebookApp to ServerApp. This config will be passed to ServerApp. Be sure to update your config before our next release.
[W 2021-08-20 13:50:56.940 NotebookApp] 'port' has moved from NotebookApp to ServerApp. This config will be passed to ServerApp. Be sure to update your config before our next release.
[I 2021-08-20 13:50:56.947 ServerApp] Writing Jupyter server cookie secret to /home/jovyan/.local/share/jupyter/runtime/jupyter_cookie_secret
[I 2021-08-20 13:50:57.097 ServerApp] nbclassic | extension was successfully linked.
[I 2021-08-20 13:50:57.117 ServerApp] nbclassic | extension was successfully loaded.
[I 2021-08-20 13:50:57.118 LabApp] JupyterLab extension loaded from /opt/conda/lib/python3.9/site-packages/jupyterlab
[I 2021-08-20 13:50:57.118 LabApp] JupyterLab application directory is /opt/conda/share/jupyter/lab
[I 2021-08-20 13:50:57.121 ServerApp] jupyterlab | extension was successfully loaded.
[I 2021-08-20 13:50:57.121 ServerApp] Serving notebooks from local directory: /home/jovyan
[I 2021-08-20 13:50:57.121 ServerApp] Jupyter Server 1.10.2 is running at:
[I 2021-08-20 13:50:57.121 ServerApp] http://jupyter-gcb:8090/lab?token=e75f40fdfcbe7318695b89f1326be0dad3cf0df333f1ca02
[I 2021-08-20 13:50:57.121 ServerApp]  or http://127.0.0.1:8090/lab?token=e75f40fdfcbe7318695b89f1326be0dad3cf0df333f1ca02
[I 2021-08-20 13:50:57.121 ServerApp] Use Control-C to stop this server and shut down all kernels (twice to skip confirmation).
[C 2021-08-20 13:50:57.124 ServerApp]

    To access the server, open this file in a browser:
        file:///home/jovyan/.local/share/jupyter/runtime/jpserver-1-open.html
    Or copy and paste one of these URLs:
        http://jupyter-gcb:8090/lab?token=token
     or http://127.0.0.1:8090/lab?token=token
[I 2021-08-20 13:52:48.818 LabApp] Build is up to date
[I 2021-08-20 13:52:50.469 ServerApp] Creating new notebook in
[I 2021-08-20 13:52:50.481 ServerApp] Writing notebook-signing key to /home/jovyan/.local/share/jupyter/notebook_secret
[I 2021-08-20 13:52:51.002 ServerApp] Kernel started: 46cd11b5-e5a2-41ef-99fd-e6aa432b1a9c
[I 2021-08-20 13:52:51.504 ServerApp] Starting buffering for 46cd11b5-e5a2-41ef-99fd-e6aa432b1a9c:7f8da3cb-37cc-45d3-acd3-6f43f958a923
[I 2021-08-20 13:54:52.247 ServerApp] Saving file at /Untitled.ipynb
[I 2021-08-20 14:26:34.830 ServerApp] Starting buffering for 46cd11b5-e5a2-41ef-99fd-e6aa432b1a9c:9cc8bc73-a900-4b4d-8d37-3a5457491353

I've never seen anything like this issue and I'm not sure how to proceed. I don't know if it is a docker issue, a kepler issue, or an NPM issue? If anyone has any insights, it would be greatly appreciated. For now, I cannot get Kepler to run on Jupyter unless I build my container manually.

mathbunnyru commented 2 years ago

@amerenda sorry for not being able to help you.

Could you please check, that this was fixed for the latest versions and works well now?

mathbunnyru commented 2 years ago

@amerenda I'm closing this issue for now. Please, reopen it and check with latest versions that you still have this problem.