jupyter / docker-stacks

Ready-to-run Docker images containing Jupyter applications
https://jupyter-docker-stacks.readthedocs.io
Other
8.01k stars 2.99k forks source link

Unable to rebuild working datascience-notebook? #668

Closed dirkcgrunwald closed 6 years ago

dirkcgrunwald commented 6 years ago

What docker image you are using?

Example: jupyter/datascience-notebook

What complete docker command do you run to launch the container (omitting sensitive values)?

Example: docker run --rm -it -p 8888:8888 jupyter/datascience-notebook jupyter lab

What steps do you take once the container is running to reproduce the issue?

Example:

  1. Start R and run capabilities()

What do you expect to happen?

Expect to see that e.g. PNG support is enabled

What actually happens?

PNG support (and others) not enabled.

I'm trying to trouble shoot some issues that have arisen since the docker-stacks started using Bionic 18.04. In the process of that, I'm trying to re-build the datascience-notebook using the jupyter/docker-stacks repo. I go through building base-, minimal-, scipy- and then datascience- using "make build/base-notebook" etc on a Mac.

When I run R notebooks through docker run --rm -it -p 8888:8888 2339546dfd7b jupyter lab, the container logs complain:

[I 04:55:15.916 LabApp] KernelRestarter: restarting kernel (1/5), keep random ports
kernel 43164dc2-3eef-4e19-8aff-c1a7fe46f1af restarted
Error in .External2(C_X11, paste0("png::", filename), g$width, g$height,  : 
  unable to start device PNG
Calls: <Anonymous> ... evaluate -> dev.new -> do.call -> <Anonymous> -> ok_device
In addition: Warning message:
In ok_device(filename, ...) : no png support in this version of R
Execution halted

and when I check the capabilities() in the installed version of R, I see that PNG is reporting FALSE (i.e. no PNG support). If I pull the repos from 16 days ago from dockerhub, R works.

But, if I start with a clean Docker system (did a reset, etc) and do a "build-all-test" and test the resulting datascience-notebook, I get the results above.

I can't figure out what is breaking or the difference between the two because the version at https://hub.docker.com/r/jupyter/datascience-notebook/tags/ appears to have been built from the June 23rd commit that would be the most recent commit in my clone.

Any ideas why building on docker locally vs. using the dockerhub image would produce different results?

The conda packages installed are shown below.

jovyan@d1370570ab5a:~$ conda list
# packages in environment at /opt/conda:
#
# Name                    Version                   Build  Channel
alembic                   0.9.9                      py_0    conda-forge
asn1crypto                0.24.0                     py_1    conda-forge
backcall                  0.1.0                      py_0    conda-forge
beautifulsoup4            4.6.0                    py36_0    conda-forge
blas                      1.1                    openblas    conda-forge
bleach                    2.1.3                      py_0    conda-forge
bokeh                     0.12.16                  py36_0    conda-forge
bzip2                     1.0.6                         1    conda-forge
ca-certificates           2018.4.16                     0    conda-forge
cairo                     1.14.10                       0    conda-forge
certifi                   2018.4.16                py36_0    conda-forge
cffi                      1.11.5                   py36_0    conda-forge
chardet                   3.0.4                      py_1    conda-forge
cloudpickle               0.5.3                      py_0    conda-forge
conda                     4.5.6                    py36_0    conda-forge
conda-env                 2.6.0                         0    conda-forge
configurable-http-proxy   3.1.0                   node8_1    conda-forge
cryptography              2.2.1            py36hdffb7b8_1    conda-forge
curl                      7.60.0                        0    conda-forge
cycler                    0.10.0                     py_1    conda-forge
cython                    0.28.4           py36hfc679d8_0    conda-forge
dask-core                 0.18.1                     py_0    conda-forge
dbus                      1.13.0               h3a4f0e9_0    conda-forge
decorator                 4.3.0                      py_0    conda-forge
dill                      0.2.8.2                  py36_0    conda-forge
entrypoints               0.2.3                    py36_1    conda-forge
expat                     2.2.5                hfc679d8_1    conda-forge
fastcache                 1.0.2                    py36_0    conda-forge
fontconfig                2.12.6                        0    conda-forge
freetype                  2.8.1                         0    conda-forge
gettext                   0.19.8.1                      0    conda-forge
glib                      2.55.0                        0    conda-forge
gmp                       6.1.2                hfc679d8_0    conda-forge
gmpy2                     2.0.8                    py36_1    conda-forge
graphite2                 1.3.11               hfc679d8_0    conda-forge
gsl                       2.4             blas_openblas_0  [blas_openblas]  conda-forge
gst-plugins-base          1.8.0                         0    conda-forge
gstreamer                 1.8.0                         1    conda-forge
h5py                      2.7.1                    py36_3    conda-forge
harfbuzz                  1.7.6                         0    conda-forge
hdf5                      1.10.1                        2    conda-forge
html5lib                  1.0.1                      py_0    conda-forge
icu                       58.2                 hfc679d8_0    conda-forge
idna                      2.7                        py_1    conda-forge
imageio                   2.3.0                    py36_0    conda-forge
ipykernel                 4.8.2                    py36_0    conda-forge
ipython                   6.4.0                    py36_0    conda-forge
ipython_genutils          0.2.0                      py_1    conda-forge
ipywidgets                7.2.1                    py36_1    conda-forge
jedi                      0.12.0                   py36_0    conda-forge
jinja2                    2.10                     py36_0    conda-forge
jpeg                      9c                   h470a237_0    conda-forge
jsonschema                2.6.0                    py36_1    conda-forge
jupyter_client            5.2.3                      py_1    conda-forge
jupyter_core              4.4.0                      py_0    conda-forge
jupyterhub                0.8.1                    py36_0    conda-forge
jupyterlab                0.32.1                   py36_0    conda-forge
jupyterlab_launcher       0.10.5                   py36_0    conda-forge
kiwisolver                1.0.1                    py36_1    conda-forge
krb5                      1.14.6                        0    conda-forge
libedit                   3.1.20170329         haf1bffa_0    conda-forge
libffi                    3.2.1                         3    conda-forge
libgcc                    7.2.0                h69d50b8_2    defaults
libgcc-ng                 7.2.0                hdf63c60_3    defaults
libgfortran               3.0.0                         1    defaults
libgfortran-ng            7.2.0                hdf63c60_3    defaults
libiconv                  1.15                 h470a237_1    conda-forge
libpng                    1.6.34               ha92aebf_1    conda-forge
libprotobuf               3.5.2                hd28b015_1    conda-forge
libsodium                 1.0.16                        0    conda-forge
libssh2                   1.8.0                h5b517e9_2    conda-forge
libstdcxx-ng              7.2.0                hdf63c60_3    defaults
libtiff                   4.0.9                he6b73bb_1    conda-forge
libuuid                   1.0.3                         1    conda-forge
libxcb                    1.13                 h470a237_1    conda-forge
libxml2                   2.9.8                h422b904_2    conda-forge
llvmlite                  0.23.0                   py36_1    conda-forge
mako                      1.0.7                    py36_0    conda-forge
markupsafe                1.0                      py36_0    conda-forge
matplotlib                2.2.2                    py36_1    conda-forge
mistune                   0.8.3                    py36_1    conda-forge
mpc                       1.1.0                         4    conda-forge
mpfr                      3.1.5                         0    conda-forge
mpmath                    1.0.0                      py_0    conda-forge
nbconvert                 5.3.1                      py_1    conda-forge
nbformat                  4.4.0                    py36_0    conda-forge
ncurses                   6.1                           0    conda-forge
networkx                  2.1                        py_1    conda-forge
nodejs                    8.10.0                        0    conda-forge
notebook                  5.5.0                    py36_0    conda-forge
numba                     0.38.1                   py36_0    conda-forge
numexpr                   2.6.5                    py36_0    conda-forge
numpy                     1.13.3          py36_blas_openblas_201  [blas_openblas]  conda-forge
olefile                   0.45.1                     py_1    conda-forge
openblas                  0.2.20                        8    conda-forge
openssl                   1.0.2o                        0    conda-forge
packaging                 17.1                       py_0    conda-forge
pamela                    0.3.0                    py36_0    conda-forge
pandas                    0.23.3                   py36_0    conda-forge
pandoc                    2.2.1                         0    conda-forge
pandocfilters             1.4.2                    py36_0    conda-forge
pango                     1.40.14                       0    conda-forge
parso                     0.3.0                      py_0    conda-forge
patsy                     0.5.0                    py36_0    conda-forge
pcre                      8.39                          0    conda-forge
pexpect                   4.6.0                    py36_0    conda-forge
pickleshare               0.7.4                    py36_0    conda-forge
pillow                    5.2.0                    py36_0    conda-forge
pip                       9.0.3                    py36_0    conda-forge
pixman                    0.34.0                        2    conda-forge
prompt_toolkit            1.0.15                   py36_0    conda-forge
protobuf                  3.5.2                    py36_0    conda-forge
pthread-stubs             0.4                  h470a237_1    conda-forge
ptyprocess                0.6.0                    py36_0    conda-forge
pycosat                   0.6.3                    py36_0    conda-forge
pycparser                 2.18                     py36_0    conda-forge
pygments                  2.2.0                      py_1    conda-forge
pyopenssl                 18.0.0                   py36_0    conda-forge
pyparsing                 2.2.0                      py_1    conda-forge
pysocks                   1.6.8                    py36_1    conda-forge
python                    3.6.5                hc3d631a_2    defaults
python-dateutil           2.7.3                      py_0    conda-forge
python-editor             1.0.3                    py36_0    conda-forge
python-oauth2             1.0.1                    py36_0    conda-forge
pytz                      2018.5                     py_0    conda-forge
pywavelets                0.5.2                    py36_1    conda-forge
pyyaml                    3.12                     py36_1    conda-forge
pyzmq                     17.0.0                   py36_4    conda-forge
r-abind                   1.4_5                  r3.4.1_0    conda-forge
r-assertthat              0.2.0                  r3.4.1_0    conda-forge
r-backports               1.1.2                  r3.4.1_0    conda-forge
r-base                    3.4.1                hff14c37_5    conda-forge
r-base64enc               0.1_3                  r3.4.1_0    conda-forge
r-bh                      1.66.0_1               r3.4.1_0    conda-forge
r-bindr                   0.1.1                  r3.4.1_0    conda-forge
r-bindrcpp                0.2                    r3.4.1_0    conda-forge
r-bit                     1.1_12                 r3.4.1_0    conda-forge
r-bit64                   0.9_5                  r3.4.1_0    conda-forge
r-bitops                  1.0_6                  r3.4.1_0    conda-forge
r-blob                    1.1.1                  r3.4.1_0    conda-forge
r-broom                   0.4.5                    r341_0    conda-forge
r-caret                   6.0_79                 r3.4.1_0    conda-forge
r-cellranger              1.1.0                  r3.4.1_0    conda-forge
r-class                   7.3_14                 r3.4.1_0    conda-forge
r-cli                     1.0.0                  r3.4.1_0    conda-forge
r-codetools               0.2_15                 r3.4.1_0    conda-forge
r-colorspace              1.3_2                  r3.4.1_0    conda-forge
r-config                  0.2                    r3.4.1_0    conda-forge
r-crayon                  1.3.4                  r3.4.1_0    conda-forge
r-curl                    3.2                    r3.4.1_0    conda-forge
r-cvst                    0.2_1                  r3.4.1_0    conda-forge
r-dbi                     1.0.0                    r341_0    conda-forge
r-dbplyr                  1.2.1                    r341_1    conda-forge
r-ddalpha                 1.3.4                    r341_0    conda-forge
r-deoptimr                1.0_8                  r3.4.1_0    conda-forge
r-devtools                1.13.6                   r341_0    conda-forge
r-dichromat               2.0_0                  r3.4.1_0    conda-forge
r-digest                  0.6.15                 r3.4.1_0    conda-forge
r-dimred                  0.1.0                  r3.4.1_0    conda-forge
r-dplyr                   0.7.4                  r3.4.1_0    conda-forge
r-drr                     0.0.3                  r3.4.1_0    conda-forge
r-evaluate                0.10.1                 r3.4.1_0    conda-forge
r-forcats                 0.3.0                  r3.4.1_0    conda-forge
r-foreach                 1.4.4                  r3.4.1_0    conda-forge
r-forecast                8.2                    r3.4.1_0    conda-forge
r-foreign                 0.8_67                 r3.4.1_0    conda-forge
r-fracdiff                1.4_2                  r3.4.1_0    conda-forge
r-geometry                0.3_6                  r3.4.1_0    conda-forge
r-ggplot2                 2.2.1                  r3.4.1_0    conda-forge
r-git2r                   0.21.0           r341h0c37787_0    conda-forge
r-glue                    1.2.0                  r3.4.1_0    conda-forge
r-gower                   0.1.2                  r3.4.1_0    conda-forge
r-gtable                  0.2.0                  r3.4.1_0    conda-forge
r-haven                   1.1.2                    r341_0    conda-forge
r-hexbin                  1.27.2                 r3.4.1_0    conda-forge
r-highr                   0.7                      r341_0    conda-forge
r-hms                     0.3                    r3.4.1_0    conda-forge
r-htmltools               0.3.6                  r3.4.1_0    conda-forge
r-htmlwidgets             1.0                    r3.4.1_0    conda-forge
r-httpuv                  1.3.6.2                r3.4.1_0    conda-forge
r-httr                    1.3.1                  r3.4.1_0    conda-forge
r-ipred                   0.9_6                  r3.4.1_0    conda-forge
r-irdisplay               0.4.4                  r3.4.1_0    conda-forge
r-irkernel                0.8.12                   r341_0    conda-forge
r-iterators               1.0.9                  r3.4.1_0    conda-forge
r-jsonlite                1.5                    r3.4.1_0    conda-forge
r-kernlab                 0.9_25                 r3.4.1_0    conda-forge
r-kernsmooth              2.23_15                r3.4.1_0    conda-forge
r-knitr                   1.20                   r3.4.1_0    conda-forge
r-labeling                0.3                    r3.4.1_0    conda-forge
r-lattice                 0.20_34                r3.4.1_0    conda-forge
r-lava                    1.6.2                    r341_0    conda-forge
r-lazyeval                0.2.1                  r3.4.1_0    conda-forge
r-lmtest                  0.9_35                 r3.4.1_0    conda-forge
r-lubridate               1.7.4                  r3.4.1_0    conda-forge
r-magic                   1.5_6                  r3.4.1_0    conda-forge
r-magrittr                1.5                    r3.4.1_0    conda-forge
r-markdown                0.8                    r3.4.1_1    conda-forge
r-mass                    7.3_48                 r3.4.1_0    conda-forge
r-matrix                  1.2_14                 r3.4.1_0    conda-forge
r-memoise                 1.1.0                  r3.4.1_0    conda-forge
r-mime                    0.5                    r3.4.1_0    conda-forge
r-mnormt                  1.5_5                  r3.4.1_0    conda-forge
r-modelmetrics            1.1.0                  r3.4.1_0    conda-forge
r-modelr                  0.1.2                    r341_0    conda-forge
r-munsell                 0.5.0                    r341_0    conda-forge
r-nlme                    3.1_131                r3.4.1_0    conda-forge
r-nnet                    7.3_12                 r3.4.1_0    conda-forge
r-numderiv                2016.8_1               r3.4.1_0    conda-forge
r-nycflights13            0.2.2                  r3.4.1_0    conda-forge
r-openssl                 1.0.1                  r3.4.1_0    conda-forge
r-pbdzmq                  0.3_2                  r3.4.1_0    conda-forge
r-pillar                  1.2.2            r341h6115d3f_1    conda-forge
r-pkgconfig               2.0.1                  r3.4.1_0    conda-forge
r-plogr                   0.1_1                  r3.4.1_0    conda-forge
r-pls                     2.6_0                  r3.4.1_0    conda-forge
r-plyr                    1.8.4                  r3.4.1_0    conda-forge
r-prettyunits             1.0.2                  r3.4.1_0    conda-forge
r-prodlim                 2018.04.18             r3.4.1_0    conda-forge
r-psych                   1.8.4                    r341_0    conda-forge
r-purrr                   0.2.4                  r3.4.1_0    conda-forge
r-quadprog                1.5_5                  r3.4.1_0    conda-forge
r-r6                      2.2.2                  r3.4.1_0    conda-forge
r-randomforest            4.6_12                 r3.4.1_0    conda-forge
r-rappdirs                0.3.1                  r3.4.1_0    conda-forge
r-rcolorbrewer            1.1_2                  r3.4.1_0    conda-forge
r-rcpp                    0.12.16                r3.4.1_0    conda-forge
r-rcpparmadillo           0.8.500.0              r3.4.1_0    conda-forge
r-rcpproll                0.2.2                  r3.4.1_0    conda-forge
r-rcurl                   1.95_4.8               r3.4.1_0    conda-forge
r-readr                   1.1.1                  r3.4.1_0    conda-forge
r-readxl                  1.1.0                    r341_0    conda-forge
r-recipes                 0.1.3                    r341_0    conda-forge
r-rematch                 1.0.1                  r3.4.1_0    conda-forge
r-repr                    0.15                     r341_0    conda-forge
r-reshape2                1.4.3                  r3.4.1_0    conda-forge
r-rlang                   0.2.1                    r341_0    conda-forge
r-rmarkdown               1.8                    r3.4.1_0    conda-forge
r-robustbase              0.92_7                 r3.4.1_0    conda-forge
r-rpart                   4.1_13                 r3.4.1_0    conda-forge
r-rprojroot               1.3_2                  r3.4.1_0    conda-forge
r-rsqlite                 2.0                    r3.4.1_0    conda-forge
r-rstudioapi              0.7                    r3.4.1_0    conda-forge
r-rvest                   0.3.2                  r3.4.1_0    conda-forge
r-scales                  0.5.0                  r3.4.1_0    conda-forge
r-selectr                 0.3_1                  r3.4.1_0    conda-forge
r-sfsmisc                 1.1_1                  r3.4.1_0    conda-forge
r-shiny                   1.0.5                  r3.4.1_0    conda-forge
r-sourcetools             0.1.6                  r3.4.1_0    conda-forge
r-sparklyr                0.7.0                  r3.4.1_0    conda-forge
r-squarem                 2017.10_1              r3.4.1_0    conda-forge
r-stringi                 1.2.3                    r341_0    conda-forge
r-stringr                 1.3.1                    r341_0    conda-forge
r-survival                2.40_1                 r3.4.1_0    conda-forge
r-tibble                  1.4.2                  r3.4.1_0    conda-forge
r-tidyr                   0.8.1                    r341_0    conda-forge
r-tidyselect              0.2.4                  r3.4.1_0    conda-forge
r-tidyverse               1.1.1                  r3.4.1_0    conda-forge
r-timedate                3012.100               r3.4.1_0    conda-forge
r-tseries                 0.10_40                r3.4.1_0    conda-forge
r-utf8                    1.1.3                  r3.4.1_0    conda-forge
r-uuid                    0.1_2                  r3.4.1_0    conda-forge
r-viridislite             0.3.0                  r3.4.1_0    conda-forge
r-whisker                 0.3_2                  r3.4.1_0    conda-forge
r-withr                   2.1.1                  r3.4.1_0    conda-forge
r-xml2                    1.2.0                    r341_0    conda-forge
r-xtable                  1.8_2                  r3.4.1_0    conda-forge
r-yaml                    2.1.18                 r3.4.1_0    conda-forge
r-zoo                     1.8_1                  r3.4.1_0    conda-forge
readline                  7.0                  ha6073c6_4    defaults
requests                  2.19.1                     py_1    conda-forge
rpy2                      2.8.5              py36r3.4.1_0    conda-forge
ruamel_yaml               0.15.42          py36h470a237_0    conda-forge
scikit-image              0.14.0           py36hfc679d8_1    conda-forge
scikit-learn              0.19.1          py36_blas_openblas_201  [blas_openblas]  conda-forge
scipy                     1.1.0           py36_blas_openblas_200  [blas_openblas]  conda-forge
seaborn                   0.8.1                    py36_0    conda-forge
send2trash                1.5.0                      py_0    conda-forge
setuptools                40.0.0                   py36_0    conda-forge
simplegeneric             0.8.1                      py_1    conda-forge
singledispatch            3.4.0.3                  py36_0    conda-forge
sip                       4.18                     py36_1    conda-forge
six                       1.11.0                   py36_1    conda-forge
sqlalchemy                1.2.9                    py36_0    conda-forge
sqlite                    3.24.0               h2f33b56_0    conda-forge
statsmodels               0.9.0                    py36_0    conda-forge
sympy                     1.1.1                    py36_0    conda-forge
terminado                 0.8.1                    py36_0    conda-forge
testpath                  0.3.1                    py36_0    conda-forge
tk                        8.6.7                         0    conda-forge
toolz                     0.9.0                      py_0    conda-forge
tornado                   5.0.2                    py36_0    conda-forge
traitlets                 4.3.2                    py36_0    conda-forge
urllib3                   1.23                     py36_0    conda-forge
vincent                   0.4.4                      py_1    conda-forge
wcwidth                   0.1.7                      py_1    conda-forge
webencodings              0.5                      py36_0    conda-forge
wheel                     0.31.1                   py36_0    conda-forge
widgetsnbextension        3.2.1                    py36_0    conda-forge
xlrd                      1.1.0                      py_2    conda-forge
xorg-kbproto              1.0.7                h470a237_2    conda-forge
xorg-libice               1.0.9                h470a237_3    conda-forge
xorg-libsm                1.2.2                h470a237_3    conda-forge
xorg-libx11               1.6.5                h470a237_1    conda-forge
xorg-libxau               1.0.8                h470a237_5    conda-forge
xorg-libxdmcp             1.1.2                h470a237_6    conda-forge
xorg-libxext              1.3.3                h470a237_3    conda-forge
xorg-libxrender           0.9.10               h470a237_1    conda-forge
xorg-renderproto          0.11.1               h470a237_2    conda-forge
xorg-xextproto            7.3.0                h470a237_2    conda-forge
xorg-xproto               7.0.31               h470a237_7    conda-forge
xz                        5.2.3                         0    conda-forge
yaml                      0.1.7                         0    conda-forge
zeromq                    4.2.5                hfc679d8_3    conda-forge
zlib                      1.2.11               h470a237_3    conda-forge
jovyan@d1370570ab5a:~$
nazamy commented 6 years ago

Yup, I'm having the same issue with R

parente commented 6 years ago

We try to pin the minor versions of packages listed in the dockerfiles here. If new patch releases or new builds of those packages become available between builds, the results will not be the same. If the packages we list do not pin their dependencies similarly (and most do not), it's possible for those dependency packages to change across builds as well.

PNG support for R has been a problem here in the past (e.g., #210). It's possible there is a compatibility problem again among R, IRkernel, libpng, etc.

parente commented 6 years ago

I believe #679 has the links upstream to the R package problems from conda-forge that are causing this problem. I'm going to close out this issue as a duplicate since the other has more direct information about the problem.