Launching Binder is _really_ slow.

kmdalton commented 2 years ago
This is way outside of my wheelhouse, but I clicked on our binder link for the first time, and it takes a crazy amount of time (8-10 minutes!) to launch. On our end, I think all we're asking for Binder to do is run pip install reciprocalspaceship[dev]. This command installs a lot of stuff. Notably, it installs PyTorch for the robust merging example. I suspect this is what takes so long, but it could certainly be other packages.
I would suggest it is worth some time to pair down the dependencies. In the case of PyTorch, we could defer the install to the first cell of the example notebook.
I'd love some help from @ianhi on this one. He certainly knows this stuff better than I.
Here is a partial log from a binder launch which may be helpful:
Picked Git content provider.
Cloning into '/tmp/repo2docker4xoq9m89'...
HEAD is now at 7781360 Bump version to 0.9.18
Building conda environment for python=3.7Using PythonBuildPack builder
Building conda environment for python=3.7Building conda environment for python=3.7Step 1/48 : FROM buildpack-deps:bionic
 ---> 72ccd8e28f8d
Step 2/48 : ENV DEBIAN_FRONTEND=noninteractive
 ---> Using cache
 ---> 915bfe954aed
Step 3/48 : RUN apt-get -qq update &&     apt-get -qq install --yes --no-install-recommends locales > /dev/null &&     apt-get -qq purge &&     apt-get -qq clean &&     rm -rf /var/lib/apt/lists/*
 ---> Using cache
 ---> 382a4bfa36dd
Step 4/48 : RUN echo "en_US.UTF-8 UTF-8" > /etc/locale.gen &&     locale-gen
 ---> Using cache
 ---> 23cc551494ea
Step 5/48 : ENV LC_ALL en_US.UTF-8
 ---> Using cache
 ---> ce16adc179fd
Step 6/48 : ENV LANG en_US.UTF-8
 ---> Using cache
 ---> 1443ecbb572c
Step 7/48 : ENV LANGUAGE en_US.UTF-8
 ---> Using cache
 ---> 67fd1928a7e1
Step 8/48 : ENV SHELL /bin/bash
 ---> Using cache
 ---> 0492f760ec33
Step 9/48 : ARG NB_USER
 ---> Using cache
 ---> 0fa8275551af
Step 10/48 : ARG NB_UID
 ---> Using cache
 ---> 9283ec23cac9
Step 11/48 : ENV USER ${NB_USER}
 ---> Using cache
 ---> e92aaa5352f0
Step 12/48 : ENV HOME /home/${NB_USER}
 ---> Using cache
 ---> 6f44454fabec
Step 13/48 : RUN groupadd         --gid ${NB_UID}         ${NB_USER} &&     useradd         --comment "Default user"         --create-home         --gid ${NB_UID}         --no-log-init         --shell /bin/bash         --uid ${NB_UID}         ${NB_USER}
 ---> Using cache
 ---> 0ee4d890aa6c
Step 14/48 : RUN apt-get -qq update &&     apt-get -qq install --yes --no-install-recommends        less        unzip        > /dev/null &&     apt-get -qq purge &&     apt-get -qq clean &&     rm -rf /var/lib/apt/lists/*
 ---> Using cache
 ---> bd296a1594c5
Step 15/48 : EXPOSE 8888
 ---> Using cache
 ---> 6d6578ba9b76
Step 16/48 : ENV APP_BASE /srv
 ---> Using cache
 ---> 92901276e26d
Step 17/48 : ENV CONDA_DIR ${APP_BASE}/conda
 ---> Using cache
 ---> c61852980ae7
Step 18/48 : ENV NB_PYTHON_PREFIX ${CONDA_DIR}/envs/notebook
 ---> Using cache
 ---> 4f44e70e3668
Step 19/48 : ENV NPM_DIR ${APP_BASE}/npm
 ---> Using cache
 ---> 9bae366c7f81
Step 20/48 : ENV NPM_CONFIG_GLOBALCONFIG ${NPM_DIR}/npmrc
 ---> Using cache
 ---> 8fe62e0295b7
Step 21/48 : ENV NB_ENVIRONMENT_FILE /tmp/env/environment.lock
 ---> Using cache
 ---> a7dee12cf999
Step 22/48 : ENV KERNEL_PYTHON_PREFIX ${NB_PYTHON_PREFIX}
 ---> Using cache
 ---> 8b2229129a3d
Step 23/48 : ENV PATH ${NB_PYTHON_PREFIX}/bin:${CONDA_DIR}/bin:${NPM_DIR}/bin:${PATH}
 ---> Using cache
 ---> 4c9bcaf823ba
Step 24/48 : COPY --chown=1000:1000 build_script_files/-2fusr-2flib-2fpython3-2e8-2fsite-2dpackages-2frepo2docker-2fbuildpacks-2fconda-2factivate-2dconda-2esh-391af5 /etc/profile.d/activate-conda.sh
 ---> Using cache
 ---> f8d2890e201b
Step 25/48 : COPY --chown=1000:1000 build_script_files/-2fusr-2flib-2fpython3-2e8-2fsite-2dpackages-2frepo2docker-2fbuildpacks-2fconda-2fenvironment-2epy-2d3-2e7-2elock-4f1154 /tmp/env/environment.lock
 ---> Using cache
 ---> c1410cdb5a70
Step 26/48 : COPY --chown=1000:1000 build_script_files/-2fusr-2flib-2fpython3-2e8-2fsite-2dpackages-2frepo2docker-2fbuildpacks-2fconda-2finstall-2dminiforge-2ebash-514214 /tmp/install-miniforge.bash
 ---> Using cache
 ---> 44936c7cb078
Step 27/48 : RUN TIMEFORMAT='time: %3R' bash -c 'time /tmp/install-miniforge.bash' && rm -rf /tmp/install-miniforge.bash /tmp/env
 ---> Using cache
 ---> 4ab657a80f7e
Step 28/48 : RUN mkdir -p ${NPM_DIR} && chown -R ${NB_USER}:${NB_USER} ${NPM_DIR}
 ---> Using cache
 ---> 01840c1deeaf
Step 29/48 : ARG REPO_DIR=${HOME}
 ---> Using cache
 ---> b440807ab159
Step 30/48 : ENV REPO_DIR ${REPO_DIR}
 ---> Using cache
 ---> f1d0395dc84d
Step 31/48 : WORKDIR ${REPO_DIR}
 ---> Using cache
 ---> 11e14be612ee
Step 32/48 : RUN chown ${NB_USER}:${NB_USER} ${REPO_DIR}
 ---> Using cache
 ---> 4baa295e9a1c
Step 33/48 : ENV PATH ${HOME}/.local/bin:${REPO_DIR}/.local/bin:${PATH}
 ---> Using cache
 ---> 8303e9277f1c
Step 34/48 : ENV CONDA_DEFAULT_ENV ${KERNEL_PYTHON_PREFIX}
 ---> Using cache
 ---> 4c3893acd0bd
Step 35/48 : COPY --chown=1000:1000 src/ ${REPO_DIR}
 ---> d5bd5f9ab584
Step 36/48 : USER ${NB_USER}
 ---> [Warning] Your kernel does not support swap limit capabilities or the cgroup is not mounted. Memory limited without swap.
 ---> Running in 54852568430a
Removing intermediate container 54852568430a
 ---> 1191068b9b5d
Step 37/48 : RUN ${KERNEL_PYTHON_PREFIX}/bin/pip install --no-cache-dir .
 ---> [Warning] Your kernel does not support swap limit capabilities or the cgroup is not mounted. Memory limited without swap.
 ---> Running in 05f10e1eaab0
Processing /home/jovyan
  Preparing metadata (setup.py): started
  Preparing metadata (setup.py): finished with status 'done'
Collecting gemmi<=0.5.1,>=0.4.2
  Downloading gemmi-0.5.1-cp37-cp37m-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (3.0 MB)
Collecting pandas<=1.3.5,>=1.2.0
  Downloading pandas-1.3.5-cp37-cp37m-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (11.3 MB)
Collecting numpy
  Downloading numpy-1.21.5-cp37-cp37m-manylinux_2_12_x86_64.manylinux2010_x86_64.whl (15.7 MB)
Collecting scipy
  Downloading scipy-1.7.3-cp37-cp37m-manylinux_2_12_x86_64.manylinux2010_x86_64.whl (38.1 MB)
Requirement already satisfied: ipython in /srv/conda/envs/notebook/lib/python3.7/site-packages (from reciprocalspaceship==0.9.18) (7.30.1)
Requirement already satisfied: pytz>=2017.3 in /srv/conda/envs/notebook/lib/python3.7/site-packages (from pandas<=1.3.5,>=1.2.0->reciprocalspaceship==0.9.18) (2021.3)
Requirement already satisfied: python-dateutil>=2.7.3 in /srv/conda/envs/notebook/lib/python3.7/site-packages (from pandas<=1.3.5,>=1.2.0->reciprocalspaceship==0.9.18) (2.8.2)
Requirement already satisfied: setuptools>=18.5 in /srv/conda/envs/notebook/lib/python3.7/site-packages (from ipython->reciprocalspaceship==0.9.18) (60.0.4)
Requirement already satisfied: decorator in /srv/conda/envs/notebook/lib/python3.7/site-packages (from ipython->reciprocalspaceship==0.9.18) (5.1.0)
Requirement already satisfied: traitlets>=4.2 in /srv/conda/envs/notebook/lib/python3.7/site-packages (from ipython->reciprocalspaceship==0.9.18) (5.1.1)
Requirement already satisfied: backcall in /srv/conda/envs/notebook/lib/python3.7/site-packages (from ipython->reciprocalspaceship==0.9.18) (0.2.0)
Requirement already satisfied: pygments in /srv/conda/envs/notebook/lib/python3.7/site-packages (from ipython->reciprocalspaceship==0.9.18) (2.10.0)
Requirement already satisfied: jedi>=0.16 in /srv/conda/envs/notebook/lib/python3.7/site-packages (from ipython->reciprocalspaceship==0.9.18) (0.18.1)
Requirement already satisfied: pexpect>4.3 in /srv/conda/envs/notebook/lib/python3.7/site-packages (from ipython->reciprocalspaceship==0.9.18) (4.8.0)
Requirement already satisfied: matplotlib-inline in /srv/conda/envs/notebook/lib/python3.7/site-packages (from ipython->reciprocalspaceship==0.9.18) (0.1.3)
Requirement already satisfied: pickleshare in /srv/conda/envs/notebook/lib/python3.7/site-packages (from ipython->reciprocalspaceship==0.9.18) (0.7.5)
Requirement already satisfied: prompt-toolkit!=3.0.0,!=3.0.1,<3.1.0,>=2.0.0 in /srv/conda/envs/notebook/lib/python3.7/site-packages (from ipython->reciprocalspaceship==0.9.18) (3.0.24)
Requirement already satisfied: parso<0.9.0,>=0.8.0 in /srv/conda/envs/notebook/lib/python3.7/site-packages (from jedi>=0.16->ipython->reciprocalspaceship==0.9.18) (0.8.3)
Requirement already satisfied: ptyprocess>=0.5 in /srv/conda/envs/notebook/lib/python3.7/site-packages (from pexpect>4.3->ipython->reciprocalspaceship==0.9.18) (0.7.0)
Requirement already satisfied: wcwidth in /srv/conda/envs/notebook/lib/python3.7/site-packages (from prompt-toolkit!=3.0.0,!=3.0.1,<3.1.0,>=2.0.0->ipython->reciprocalspaceship==0.9.18) (0.2.5)
Requirement already satisfied: six>=1.5 in /srv/conda/envs/notebook/lib/python3.7/site-packages (from python-dateutil>=2.7.3->pandas<=1.3.5,>=1.2.0->reciprocalspaceship==0.9.18) (1.16.0)
Building wheels for collected packages: reciprocalspaceship
  Building wheel for reciprocalspaceship (setup.py): started
  Building wheel for reciprocalspaceship (setup.py): finished with status 'done'
  Created wheel for reciprocalspaceship: filename=reciprocalspaceship-0.9.18-py3-none-any.whl size=68921 sha256=cefc203a0593825f967eeff3511e8c362c64247e6eaff0dfc0730ff01e1a47bd
  Stored in directory: /tmp/pip-ephem-wheel-cache-1b6sliwd/wheels/24/67/61/461d47532c7e3b6048f03e8f0e3c2ddb1976b17163d48f9fe9
Successfully built reciprocalspaceship
Installing collected packages: numpy, scipy, pandas, gemmi, reciprocalspaceship
Successfully installed gemmi-0.5.1 numpy-1.21.5 pandas-1.3.5 reciprocalspaceship-0.9.18 scipy-1.7.3
Removing intermediate container 05f10e1eaab0
 ---> 9b8a5f6ddc67
Step 38/48 : LABEL repo2docker.ref="7781360585814dbbac9ba0628fb2fbc974a964fb"
 ---> [Warning] Your kernel does not support swap limit capabilities or the cgroup is not mounted. Memory limited without swap.
 ---> Running in 3a4d023a8819
Removing intermediate container 3a4d023a8819
 ---> 6e4e4f0325d0
Step 39/48 : LABEL repo2docker.repo="https://github.com/Hekstra-Lab/reciprocalspaceship"
 ---> [Warning] Your kernel does not support swap limit capabilities or the cgroup is not mounted. Memory limited without swap.
 ---> Running in 22531833e814
Removing intermediate container 22531833e814
 ---> 46ad036a4080
Step 40/48 : LABEL repo2docker.version="2021.08.0+78.g4352535"
 ---> [Warning] Your kernel does not support swap limit capabilities or the cgroup is not mounted. Memory limited without swap.
 ---> Running in fe8f9fe7819f
Removing intermediate container fe8f9fe7819f
 ---> e0e80f22843d
Step 41/48 : USER ${NB_USER}
 ---> [Warning] Your kernel does not support swap limit capabilities or the cgroup is not mounted. Memory limited without swap.
 ---> Running in e87f3615d067
Removing intermediate container e87f3615d067
 ---> 629e656d9a07
Step 42/48 : RUN chmod +x postBuild
 ---> [Warning] Your kernel does not support swap limit capabilities or the cgroup is not mounted. Memory limited without swap.
 ---> Running in 2e7bfb1cba7d
Removing intermediate container 2e7bfb1cba7d
 ---> 233f67c9a99e
Step 43/48 : RUN ./postBuild
 ---> [Warning] Your kernel does not support swap limit capabilities or the cgroup is not mounted. Memory limited without swap.
 ---> Running in a519284bfcd0
Requirement already satisfied: reciprocalspaceship[dev] in /srv/conda/envs/notebook/lib/python3.7/site-packages (0.9.18)
Requirement already satisfied: ipython in /srv/conda/envs/notebook/lib/python3.7/site-packages (from reciprocalspaceship[dev]) (7.30.1)
Requirement already satisfied: pandas<=1.3.5,>=1.2.0 in /srv/conda/envs/notebook/lib/python3.7/site-packages (from reciprocalspaceship[dev]) (1.3.5)
Requirement already satisfied: gemmi<=0.5.1,>=0.4.2 in /srv/conda/envs/notebook/lib/python3.7/site-packages (from reciprocalspaceship[dev]) (0.5.1)
Requirement already satisfied: scipy in /srv/conda/envs/notebook/lib/python3.7/site-packages (from reciprocalspaceship[dev]) (1.7.3)
Requirement already satisfied: numpy in /srv/conda/envs/notebook/lib/python3.7/site-packages (from reciprocalspaceship[dev]) (1.21.5)
Collecting jupyter
  Downloading jupyter-1.0.0-py2.py3-none-any.whl (2.7 kB)
Collecting torch
  Downloading torch-1.10.1-cp37-cp37m-manylinux1_x86_64.whl (881.9 MB)
Collecting sphinx-rtd-theme
  Downloading sphinx_rtd_theme-1.0.0-py2.py3-none-any.whl (2.8 MB)
Collecting pytest
  Downloading pytest-6.2.5-py3-none-any.whl (280 kB)
Collecting pytest-cov
  Downloading pytest_cov-3.0.0-py3-none-any.whl (20 kB)
Collecting matplotlib
  Downloading matplotlib-3.5.1-cp37-cp37m-manylinux_2_5_x86_64.manylinux1_x86_64.whl (11.2 MB)
Collecting nbsphinx
  Downloading nbsphinx-0.8.8-py3-none-any.whl (25 kB)
Collecting seaborn
  Downloading seaborn-0.11.2-py3-none-any.whl (292 kB)
Collecting autodocsumm
  Downloading autodocsumm-0.2.7.tar.gz (43 kB)
  Preparing metadata (setup.py): started
  Preparing metadata (setup.py): finished with status 'done'
Collecting pytest-xdist
  Downloading pytest_xdist-2.5.0-py3-none-any.whl (41 kB)
Collecting sphinx
  Downloading Sphinx-4.4.0-py3-none-any.whl (3.1 MB)
Collecting tqdm
  Downloading tqdm-4.62.3-py2.py3-none-any.whl (76 kB)
Collecting scikit-image
  Downloading scikit_image-0.19.1-cp37-cp37m-manylinux_2_12_x86_64.manylinux2010_x86_64.whl (13.3 MB)
Collecting sphinx-panels
  Downloading sphinx_panels-0.6.0-py3-none-any.whl (87 kB)
Collecting sphinxcontrib-autoprogram
  Downloading sphinxcontrib_autoprogram-0.1.7-py2.py3-none-any.whl (8.7 kB)
Collecting celluloid
  Downloading celluloid-0.2.0-py3-none-any.whl (5.4 kB)
Requirement already satisfied: python-dateutil>=2.7.3 in /srv/conda/envs/notebook/lib/python3.7/site-packages (from pandas<=1.3.5,>=1.2.0->reciprocalspaceship[dev]) (2.8.2)
Requirement already satisfied: pytz>=2017.3 in /srv/conda/envs/notebook/lib/python3.7/site-packages (from pandas<=1.3.5,>=1.2.0->reciprocalspaceship[dev]) (2021.3)
Collecting sphinxcontrib-htmlhelp>=2.0.0
  Downloading sphinxcontrib_htmlhelp-2.0.0-py2.py3-none-any.whl (100 kB)
Requirement already satisfied: Jinja2>=2.3 in /srv/conda/envs/notebook/lib/python3.7/site-packages (from sphinx->reciprocalspaceship[dev]) (3.0.3)
Collecting sphinxcontrib-applehelp
  Downloading sphinxcontrib_applehelp-1.0.2-py2.py3-none-any.whl (121 kB)
Collecting snowballstemmer>=1.1
  Downloading snowballstemmer-2.2.0-py2.py3-none-any.whl (93 kB)
Collecting sphinxcontrib-qthelp
  Downloading sphinxcontrib_qthelp-1.0.3-py2.py3-none-any.whl (90 kB)
Collecting docutils<0.18,>=0.14
  Downloading docutils-0.17.1-py2.py3-none-any.whl (575 kB)
Requirement already satisfied: importlib-metadata>=4.4 in /srv/conda/envs/notebook/lib/python3.7/site-packages (from sphinx->reciprocalspaceship[dev]) (4.10.0)
Requirement already satisfied: Pygments>=2.0 in /srv/conda/envs/notebook/lib/python3.7/site-packages (from sphinx->reciprocalspaceship[dev]) (2.10.0)
Collecting sphinxcontrib-devhelp
  Downloading sphinxcontrib_devhelp-1.0.2-py2.py3-none-any.whl (84 kB)
Requirement already satisfied: requests>=2.5.0 in /srv/conda/envs/notebook/lib/python3.7/site-packages (from sphinx->reciprocalspaceship[dev]) (2.26.0)
Collecting sphinxcontrib-jsmath
  Downloading sphinxcontrib_jsmath-1.0.1-py2.py3-none-any.whl (5.1 kB)
Requirement already satisfied: packaging in /srv/conda/envs/notebook/lib/python3.7/site-packages (from sphinx->reciprocalspaceship[dev]) (21.3)
Collecting alabaster<0.8,>=0.7
  Downloading alabaster-0.7.12-py2.py3-none-any.whl (14 kB)
Requirement already satisfied: babel>=1.3 in /srv/conda/envs/notebook/lib/python3.7/site-packages (from sphinx->reciprocalspaceship[dev]) (2.9.1)
Collecting imagesize
  Downloading imagesize-1.3.0-py2.py3-none-any.whl (5.2 kB)
Collecting sphinxcontrib-serializinghtml>=1.1.5
  Downloading sphinxcontrib_serializinghtml-1.1.5-py2.py3-none-any.whl (94 kB)
Requirement already satisfied: backcall in /srv/conda/envs/notebook/lib/python3.7/site-packages (from ipython->reciprocalspaceship[dev]) (0.2.0)
Requirement already satisfied: decorator in /srv/conda/envs/notebook/lib/python3.7/site-packages (from ipython->reciprocalspaceship[dev]) (5.1.0)
Requirement already satisfied: pickleshare in /srv/conda/envs/notebook/lib/python3.7/site-packages (from ipython->reciprocalspaceship[dev]) (0.7.5)
Requirement already satisfied: jedi>=0.16 in /srv/conda/envs/notebook/lib/python3.7/site-packages (from ipython->reciprocalspaceship[dev]) (0.18.1)
Requirement already satisfied: matplotlib-inline in /srv/conda/envs/notebook/lib/python3.7/site-packages (from ipython->reciprocalspaceship[dev]) (0.1.3)
Requirement already satisfied: setuptools>=18.5 in /srv/conda/envs/notebook/lib/python3.7/site-packages (from ipython->reciprocalspaceship[dev]) (60.0.4)
Requirement already satisfied: traitlets>=4.2 in /srv/conda/envs/notebook/lib/python3.7/site-packages (from ipython->reciprocalspaceship[dev]) (5.1.1)
Requirement already satisfied: prompt-toolkit!=3.0.0,!=3.0.1,<3.1.0,>=2.0.0 in /srv/conda/envs/notebook/lib/python3.7/site-packages (from ipython->reciprocalspaceship[dev]) (3.0.24)
Requirement already satisfied: pexpect>4.3 in /srv/conda/envs/notebook/lib/python3.7/site-packages (from ipython->reciprocalspaceship[dev]) (4.8.0)
Requirement already satisfied: ipywidgets in /srv/conda/envs/notebook/lib/python3.7/site-packages (from jupyter->reciprocalspaceship[dev]) (7.6.3)
Requirement already satisfied: ipykernel in /srv/conda/envs/notebook/lib/python3.7/site-packages (from jupyter->reciprocalspaceship[dev]) (6.6.0)
Requirement already satisfied: notebook in /srv/conda/envs/notebook/lib/python3.7/site-packages (from jupyter->reciprocalspaceship[dev]) (6.3.0)
Requirement already satisfied: nbconvert in /srv/conda/envs/notebook/lib/python3.7/site-packages (from jupyter->reciprocalspaceship[dev]) (6.0.7)
Collecting jupyter-console
  Downloading jupyter_console-6.4.0-py3-none-any.whl (22 kB)
Collecting qtconsole
  Downloading qtconsole-5.2.2-py3-none-any.whl (120 kB)
Collecting fonttools>=4.22.0
  Downloading fonttools-4.28.5-py3-none-any.whl (890 kB)
Requirement already satisfied: pyparsing>=2.2.1 in /srv/conda/envs/notebook/lib/python3.7/site-packages (from matplotlib->reciprocalspaceship[dev]) (3.0.6)
Collecting pillow>=6.2.0
  Downloading Pillow-9.0.0-cp37-cp37m-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (4.3 MB)
Collecting cycler>=0.10
  Downloading cycler-0.11.0-py3-none-any.whl (6.4 kB)
Collecting kiwisolver>=1.0.1
  Downloading kiwisolver-1.3.2-cp37-cp37m-manylinux_2_5_x86_64.manylinux1_x86_64.whl (1.1 MB)
Requirement already satisfied: nbformat in /srv/conda/envs/notebook/lib/python3.7/site-packages (from nbsphinx->reciprocalspaceship[dev]) (5.1.3)
Collecting py>=1.8.2
  Downloading py-1.11.0-py2.py3-none-any.whl (98 kB)
Requirement already satisfied: attrs>=19.2.0 in /srv/conda/envs/notebook/lib/python3.7/site-packages (from pytest->reciprocalspaceship[dev]) (21.2.0)
Collecting iniconfig
  Downloading iniconfig-1.1.1-py2.py3-none-any.whl (5.0 kB)
Collecting pluggy<2.0,>=0.12
  Downloading pluggy-1.0.0-py2.py3-none-any.whl (13 kB)
Collecting toml
  Downloading toml-0.10.2-py2.py3-none-any.whl (16 kB)
Collecting coverage[toml]>=5.2.1
  Downloading coverage-6.2-cp37-cp37m-manylinux_2_5_x86_64.manylinux1_x86_64.manylinux_2_12_x86_64.manylinux2010_x86_64.whl (213 kB)
Collecting pytest-forked
  Downloading pytest_forked-1.4.0-py3-none-any.whl (4.9 kB)
Collecting execnet>=1.1
  Downloading execnet-1.9.0-py2.py3-none-any.whl (39 kB)
Collecting imageio>=2.4.1
  Downloading imageio-2.13.5-py3-none-any.whl (3.3 MB)
Collecting PyWavelets>=1.1.1
  Downloading PyWavelets-1.2.0-cp37-cp37m-manylinux_2_5_x86_64.manylinux1_x86_64.whl (6.1 MB)
Collecting networkx>=2.2
  Downloading networkx-2.6.3-py3-none-any.whl (1.9 MB)
Collecting tifffile>=2019.7.26
  Downloading tifffile-2021.11.2-py3-none-any.whl (178 kB)
Requirement already satisfied: six in /srv/conda/envs/notebook/lib/python3.7/site-packages (from sphinxcontrib-autoprogram->reciprocalspaceship[dev]) (1.16.0)
Requirement already satisfied: typing-extensions in /srv/conda/envs/notebook/lib/python3.7/site-packages (from torch->reciprocalspaceship[dev]) (4.0.1)
Collecting tomli
  Downloading tomli-2.0.0-py3-none-any.whl (12 kB)
Requirement already satisfied: zipp>=0.5 in /srv/conda/envs/notebook/lib/python3.7/site-packages (from importlib-metadata>=4.4->sphinx->reciprocalspaceship[dev]) (3.6.0)
Requirement already satisfied: parso<0.9.0,>=0.8.0 in /srv/conda/envs/notebook/lib/python3.7/site-packages (from jedi>=0.16->ipython->reciprocalspaceship[dev]) (0.8.3)
Requirement already satisfied: MarkupSafe>=2.0 in /srv/conda/envs/notebook/lib/python3.7/site-packages (from Jinja2>=2.3->sphinx->reciprocalspaceship[dev]) (2.0.1)
Requirement already satisfied: defusedxml in /srv/conda/envs/notebook/lib/python3.7/site-packages (from nbconvert->jupyter->reciprocalspaceship[dev]) (0.7.1)
Requirement already satisfied: mistune<2,>=0.8.1 in /srv/conda/envs/notebook/lib/python3.7/site-packages (from nbconvert->jupyter->reciprocalspaceship[dev]) (0.8.4)
Requirement already satisfied: testpath in /srv/conda/envs/notebook/lib/python3.7/site-packages (from nbconvert->jupyter->reciprocalspaceship[dev]) (0.5.0)
Requirement already satisfied: entrypoints>=0.2.2 in /srv/conda/envs/notebook/lib/python3.7/site-packages (from nbconvert->jupyter->reciprocalspaceship[dev]) (0.3)
Requirement already satisfied: jupyterlab-pygments in /srv/conda/envs/notebook/lib/python3.7/site-packages (from nbconvert->jupyter->reciprocalspaceship[dev]) (0.1.2)
Requirement already satisfied: jupyter-core in /srv/conda/envs/notebook/lib/python3.7/site-packages (from nbconvert->jupyter->reciprocalspaceship[dev]) (4.9.1)
Requirement already satisfied: pandocfilters>=1.4.1 in /srv/conda/envs/notebook/lib/python3.7/site-packages (from nbconvert->jupyter->reciprocalspaceship[dev]) (1.5.0)
Requirement already satisfied: bleach in /srv/conda/envs/notebook/lib/python3.7/site-packages (from nbconvert->jupyter->reciprocalspaceship[dev]) (4.1.0)
Requirement already satisfied: nbclient<0.6.0,>=0.5.0 in /srv/conda/envs/notebook/lib/python3.7/site-packages (from nbconvert->jupyter->reciprocalspaceship[dev]) (0.5.9)
Requirement already satisfied: ipython-genutils in /srv/conda/envs/notebook/lib/python3.7/site-packages (from nbformat->nbsphinx->reciprocalspaceship[dev]) (0.2.0)
Requirement already satisfied: jsonschema!=2.5.0,>=2.4 in /srv/conda/envs/notebook/lib/python3.7/site-packages (from nbformat->nbsphinx->reciprocalspaceship[dev]) (4.3.2)
Requirement already satisfied: ptyprocess>=0.5 in /srv/conda/envs/notebook/lib/python3.7/site-packages (from pexpect>4.3->ipython->reciprocalspaceship[dev]) (0.7.0)
Requirement already satisfied: wcwidth in /srv/conda/envs/notebook/lib/python3.7/site-packages (from prompt-toolkit!=3.0.0,!=3.0.1,<3.1.0,>=2.0.0->ipython->reciprocalspaceship[dev]) (0.2.5)
Requirement already satisfied: charset-normalizer~=2.0.0 in /srv/conda/envs/notebook/lib/python3.7/site-packages (from requests>=2.5.0->sphinx->reciprocalspaceship[dev]) (2.0.9)
Requirement already satisfied: certifi>=2017.4.17 in /srv/conda/envs/notebook/lib/python3.7/site-packages (from requests>=2.5.0->sphinx->reciprocalspaceship[dev]) (2021.10.8)
Requirement already satisfied: idna<4,>=2.5 in /srv/conda/envs/notebook/lib/python3.7/site-packages (from requests>=2.5.0->sphinx->reciprocalspaceship[dev]) (3.1)
Requirement already satisfied: urllib3<1.27,>=1.21.1 in /srv/conda/envs/notebook/lib/python3.7/site-packages (from requests>=2.5.0->sphinx->reciprocalspaceship[dev]) (1.26.7)
Requirement already satisfied: argcomplete>=1.12.3 in /srv/conda/envs/notebook/lib/python3.7/site-packages (from ipykernel->jupyter->reciprocalspaceship[dev]) (1.12.3)
Requirement already satisfied: debugpy<2.0,>=1.0.0 in /srv/conda/envs/notebook/lib/python3.7/site-packages (from ipykernel->jupyter->reciprocalspaceship[dev]) (1.5.1)
Requirement already satisfied: tornado<7.0,>=4.2 in /srv/conda/envs/notebook/lib/python3.7/site-packages (from ipykernel->jupyter->reciprocalspaceship[dev]) (6.1)
Requirement already satisfied: jupyter-client<8.0 in /srv/conda/envs/notebook/lib/python3.7/site-packages (from ipykernel->jupyter->reciprocalspaceship[dev]) (7.1.0)
Requirement already satisfied: widgetsnbextension~=3.5.0 in /srv/conda/envs/notebook/lib/python3.7/site-packages (from ipywidgets->jupyter->reciprocalspaceship[dev]) (3.5.2)
Requirement already satisfied: jupyterlab-widgets>=1.0.0 in /srv/conda/envs/notebook/lib/python3.7/site-packages (from ipywidgets->jupyter->reciprocalspaceship[dev]) (1.0.2)
Requirement already satisfied: pyzmq>=17 in /srv/conda/envs/notebook/lib/python3.7/site-packages (from notebook->jupyter->reciprocalspaceship[dev]) (22.3.0)
Requirement already satisfied: argon2-cffi in /srv/conda/envs/notebook/lib/python3.7/site-packages (from notebook->jupyter->reciprocalspaceship[dev]) (21.1.0)
Requirement already satisfied: prometheus-client in /srv/conda/envs/notebook/lib/python3.7/site-packages (from notebook->jupyter->reciprocalspaceship[dev]) (0.12.0)
Requirement already satisfied: terminado>=0.8.3 in /srv/conda/envs/notebook/lib/python3.7/site-packages (from notebook->jupyter->reciprocalspaceship[dev]) (0.12.1)
Requirement already satisfied: Send2Trash>=1.5.0 in /srv/conda/envs/notebook/lib/python3.7/site-packages (from notebook->jupyter->reciprocalspaceship[dev]) (1.8.0)
Collecting qtpy
  Downloading QtPy-2.0.0-py3-none-any.whl (62 kB)
Requirement already satisfied: importlib-resources>=1.4.0 in /srv/conda/envs/notebook/lib/python3.7/site-packages (from jsonschema!=2.5.0,>=2.4->nbformat->nbsphinx->reciprocalspaceship[dev]) (5.4.0)
Requirement already satisfied: pyrsistent!=0.17.0,!=0.17.1,!=0.17.2,>=0.14.0 in /srv/conda/envs/notebook/lib/python3.7/site-packages (from jsonschema!=2.5.0,>=2.4->nbformat->nbsphinx->reciprocalspaceship[dev]) (0.18.0)
Requirement already satisfied: nest-asyncio>=1.5 in /srv/conda/envs/notebook/lib/python3.7/site-packages (from jupyter-client<8.0->ipykernel->jupyter->reciprocalspaceship[dev]) (1.5.4)
Requirement already satisfied: cffi>=1.0.0 in /srv/conda/envs/notebook/lib/python3.7/site-packages (from argon2-cffi->notebook->jupyter->reciprocalspaceship[dev]) (1.15.0)
Requirement already satisfied: webencodings in /srv/conda/envs/notebook/lib/python3.7/site-packages (from bleach->nbconvert->jupyter->reciprocalspaceship[dev]) (0.5.1)
Requirement already satisfied: pycparser in /srv/conda/envs/notebook/lib/python3.7/site-packages (from cffi>=1.0.0->argon2-cffi->notebook->jupyter->reciprocalspaceship[dev]) (2.21)
Building wheels for collected packages: autodocsumm
  Building wheel for autodocsumm (setup.py): started
  Building wheel for autodocsumm (setup.py): finished with status 'done'
  Created wheel for autodocsumm: filename=autodocsumm-0.2.7-py3-none-any.whl size=13521 sha256=1d8a97a3eb5851349339a541e8d7a4d78610530a35b5b10a57e18d05b2de3c03
  Stored in directory: /home/jovyan/.cache/pip/wheels/c7/7e/cb/6102fccefbd2ca3339722fcddfa7787a88d52ddbbfbd280221
Successfully built autodocsumm
Installing collected packages: toml, py, pluggy, iniconfig, tomli, sphinxcontrib-serializinghtml, sphinxcontrib-qthelp, sphinxcontrib-jsmath, sphinxcontrib-htmlhelp, sphinxcontrib-devhelp, sphinxcontrib-applehelp, snowballstemmer, qtpy, pytest, pillow, kiwisolver, imagesize, fonttools, docutils, cycler, coverage, alabaster, tifffile, sphinx, qtconsole, PyWavelets, pytest-forked, networkx, matplotlib, jupyter-console, imageio, execnet, tqdm, torch, sphinxcontrib-autoprogram, sphinx-rtd-theme, sphinx-panels, seaborn, scikit-image, pytest-xdist, pytest-cov, nbsphinx, jupyter, celluloid, autodocsumm
Successfully installed PyWavelets-1.2.0 alabaster-0.7.12 autodocsumm-0.2.7 celluloid-0.2.0 coverage-6.2 cycler-0.11.0 docutils-0.17.1 execnet-1.9.0 fonttools-4.28.5 imageio-2.13.5 imagesize-1.3.0 iniconfig-1.1.1 jupyter-1.0.0 jupyter-console-6.4.0 kiwisolver-1.3.2 matplotlib-3.5.1 nbsphinx-0.8.8 networkx-2.6.3 pillow-9.0.0 pluggy-1.0.0 py-1.11.0 pytest-6.2.5 pytest-cov-3.0.0 pytest-forked-1.4.0 pytest-xdist-2.5.0 qtconsole-5.2.2 qtpy-2.0.0 scikit-image-0.19.1 seaborn-0.11.2 snowballstemmer-2.2.0 sphinx-4.4.0 sphinx-panels-0.6.0 sphinx-rtd-theme-1.0.0 sphinxcontrib-applehelp-1.0.2 sphinxcontrib-autoprogram-0.1.7 sphinxcontrib-devhelp-1.0.2 sphinxcontrib-htmlhelp-2.0.0 sphinxcontrib-jsmath-1.0.1 sphinxcontrib-qthelp-1.0.3 sphinxcontrib-serializinghtml-1.1.5 tifffile-2021.11.2 toml-0.10.2 tomli-2.0.0 torch-1.10.1 tqdm-4.62.3
Removing intermediate container a519284bfcd0
 ---> 42bf0708571d
Step 44/48 : ENV PYTHONUNBUFFERED=1
 ---> [Warning] Your kernel does not support swap limit capabilities or the cgroup is not mounted. Memory limited without swap.
 ---> Running in b585ae5ccf83
Removing intermediate container b585ae5ccf83
 ---> 4d56d3fbbe2d
Step 45/48 : COPY /python3-login /usr/local/bin/python3-login
 ---> 3d98ceb43ecb
Step 46/48 : COPY /repo2docker-entrypoint /usr/local/bin/repo2docker-entrypoint
 ---> 11a6f6fd2d25
Step 47/48 : ENTRYPOINT ["/usr/local/bin/repo2docker-entrypoint"]
 ---> [Warning] Your kernel does not support swap limit capabilities or the cgroup is not mounted. Memory limited without swap.
 ---> Running in 8fc78e8e4f26
Removing intermediate container 8fc78e8e4f26
 ---> 9cdcb96d4175
Step 48/48 : CMD ["jupyter", "notebook", "--ip", "0.0.0.0"]
 ---> [Warning] Your kernel does not support swap limit capabilities or the cgroup is not mounted. Memory limited without swap.
 ---> Running in 851c7348604f
Removing intermediate container 851c7348604f
 ---> 39789a85a264
{"aux": {"ID": "sha256:39789a85a2645b262f62b3bb8e887e03ae3c43efbef55692fb8103be25c09452"}}Successfully built 39789a85a264
Successfully tagged turingmybinder/binder-prod-r2d-g5b5b759-hekstra-2dlab-2dreciprocalspaceship-670595:7781360585814dbbac9ba0628fb2fbc974a964fb
Pushing image
Pushing image
Pushing image
Pushing image
Pushing image
Pushing image
Pushing image
Pushing image
Pushing image
Pushing image
Pushing image
Pushing image
Pushing image
Pushing image
Pushing image
Pushing image
Pushing image
Pushing image
Pushing image
Pushing image
Pushing image
Pushing image
Pushing image
kmdalton commented 2 years ago
Going through the examples, I have tabulated the dependencies we actually import:
reciprocalspaceship
matplotlib
seaborn
numpy
tqdm
celluloid
IPython (from IPython.disaply import HTML, Image)
pandas
gemmi
PyTorch (only in example 3)
Maybe we can make some sort of .yml file with these specific dependencies and test out the launch time? I'm looking at you, @ianhi.
JBGreisman commented 2 years ago
yeah -- I can split the [dev] dependencies between documentation / examples so that we only install the necessary ones in this context.
However, I think the main issue here is cacheing -- binder is very slow when something hasn't been set up in a while, but quite fast if it has been. I'm guessing because you just accessed it, it just loaded in ~5 seconds for me.
kmdalton commented 2 years ago
Yes, it is definitely zippy after the first run, but i think we should be able to pare down the initial build to <60s.
kmdalton commented 2 years ago
(based on other repos I've sampled).
kmdalton commented 2 years ago
Anyway --- this is outside my wheelhouse, but I figured it merited an issue.
JBGreisman commented 2 years ago
Here's a webpage with some suggestions: https://discourse.jupyter.org/t/how-to-reduce-mybinder-org-repository-startup-time/4956
I can certainly take care of making another extras_require entry in the setup.py for handling only additional packages needed for the examples. That may speed things up a bit, but @ianhi may have some other ideas -- he set up binder in the first place.
JBGreisman commented 2 years ago
I've made the simple changes that make sense. It's still not a "fast" build for the docker image, but I do think it's faster than it was. Feel free to close the issue if this seems to have improved things enough.
kmdalton commented 2 years ago
It's much faster now. Thanks!
ianhi commented 2 years ago
This can be improved further by installing all the dependencies using mamba and then only using postBuild to install reciprocalspaceship
ianhi commented 2 years ago
basically need an environment.yml file in the .binder or binder folder like here: https://github.com/jupyterlab-contrib/jupyterlab-vim/tree/master/binder
then binder will use mamba by default i think.
rs-station / reciprocalspaceship

Launching Binder is _really_ slow. #123
