search

jupyterhub / repo2docker

Turn repositories into Jupyter-enabled Docker images
https://repo2docker.readthedocs.io
BSD 3-Clause "New" or "Revised" License
1.61k stars 360 forks source link

In a Python 2 repository `python` gives you Python 3 #698

Open betatim opened 5 years ago

betatim commented 5 years ago

Bug description

In a repo with Python 2 selected via a runtime.txt the command python launches Python 3.

Expected behaviour

python should launch version 2.

How to reproduce

I think it is because of the order in which environments appear on the search path:

$ echo $PATH
/home/jovyan/.local/bin:/home/jovyan/.local/bin:/srv/conda/bin:/srv/conda/envs/kernel/bin:/srv/npm/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin

Not sure if this is a bug or merely "surprising behaviour" that needs documenting. Maybe we could write down somewhere why /srv/conda/bin does come before /srv/conda/envs/kernel/bin to help people understand why it is this way.

betatim commented 5 years ago

related/original issue https://github.com/jupyterhub/binderhub/issues/871

nuest commented 5 years ago

Activating the kernel environment switches to the correct kernel as suggested in #871

image

So :+1: on activating that in the terminal by default, see my attempt below.

Digging into the issue

AFAICS the conda environment should be activated via file conda-activate.sh: https://github.com/jupyter/repo2docker/blob/master/repo2docker/buildpacks/conda/activate-conda.sh#L17

But looking at the debug output of repo2docker locally, that file does not seem to be called. Should it be picked up by conda automatically?

Could the problem be that conda activate does not work?

jovyan@jupyter-nuest-2dpluc-5fmozambique-2difrz8msn:~$ conda activate /srv/conda/envs/notebook

CommandNotFoundError: Your shell has not been properly configured to use 'conda activate'.
To initialize your shell, run

    $ conda init <SHELL_NAME>

Currently supported shells are:
  - bash
  - fish
  - tcsh
  - xonsh
  - zsh
  - powershell

See 'conda init --help' for more information and options.

IMPORTANT: You may need to close and restart your shell after running 'conda init'.

When I run conda init bash and start a new terminal, I can activate the different envs:

(notebook) jovyan@jupyter-nuest-2dpluc-5fmozambique-2difrz8msn:~$ conda env list
# conda environments:
#
base                     /srv/conda
kernel                   /srv/conda/envs/kernel
notebook              *  /srv/conda/envs/notebook

(notebook) jovyan@jupyter-nuest-2dpluc-5fmozambique-2difrz8msn:~$ conda activate /srv/conda/envs/kernel
(kernel) jovyan@jupyter-nuest-2dpluc-5fmozambique-2difrz8msn:~$ python --version
Python 2.7.15
(kernel) jovyan@jupyter-nuest-2dpluc-5fmozambique-2difrz8msn:~$

Note that in the listing, base comes first.

Looking at the path in a repo where I encountered this issue (see https://discourse.jupyter.org/t/creating-environment-yml-for-python-2-via-binder/1609), I think there might be a bug in the third path element, which reads /srv/conda/condabin. But still, the kernel is the first Python on the path, and /srv/conda/bin also comes again later. The environment variable for the default env seems to be alright.

(kernel) jovyan@jupyter-nuest-2dpluc-5fmozambique-2difrz8msn:~$ echo $PATH
/home/jovyan/pcraster/bin:/srv/conda/envs/kernel/bin:/srv/conda/condabin:/home/jovyan/.local/bin:/home/jovyan/.local/bin:/srv/conda/envs/notebook/bin:/srv/conda/envs/kernel/bin:/srv/conda/bin:/srv/npm/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin
(kernel) jovyan@jupyter-nuest-2dpluc-5fmozambique-2difrz8msn:~$ echo $CONDA_DEFAULT_ENV
kernel
(kernel) jovyan@jupyter-nuest-2dpluc-5fmozambique-2difrz8msn:~$

Solution?

I have made the following changes, and the desired kernel is then active in the terminal rightaway (the echo is of course not strictly needed but might help debugging in the future):

https://github.com/jupyter/repo2docker/compare/master...nuest:activate-conda

Click to expand ``` daniel@f46b48213eb8:~$ python --version Python 2.7.15 daniel@f46b48213eb8:~$ conda env list # conda environments: # base /srv/conda notebook * /srv/conda/envs/notebook daniel@f46b48213eb8:~$ conda env export -n root name: root channels: - conda-forge - defaults dependencies: - asn1crypto=0.24.0=py37_0 - ca-certificates=2019.1.23=0 - certifi=2019.3.9=py37_0 - cffi=1.12.2=py37h2e261b9_1 - chardet=3.0.4=py37_1 - conda=4.6.14=py37_0 - cryptography=2.6.1=py37h1ba5d50_0 - idna=2.8=py37_0 - libedit=3.1.20181209=hc058e9b_0 - libffi=3.2.1=hd88cf55_4 - libgcc-ng=8.2.0=hdf63c60_1 - libstdcxx-ng=8.2.0=hdf63c60_1 - ncurses=6.1=he6710b0_1 - openssl=1.1.1b=h7b6447c_1 - pip=19.0.3=py37_0 - pycosat=0.6.3=py37h14c3975_0 - pycparser=2.19=py37_0 - pyopenssl=19.0.0=py37_0 - pysocks=1.6.8=py37_0 - python=3.7.3=h0371630_0 - readline=7.0=h7b6447c_5 - requests=2.21.0=py37_0 - ruamel_yaml=0.15.46=py37h14c3975_0 - setuptools=41.0.0=py37_0 - six=1.12.0=py37_0 - sqlite=3.27.2=h7b6447c_0 - tk=8.6.8=hbc83047_0 - urllib3=1.24.1=py37_0 - wheel=0.33.1=py37_0 - xz=5.2.4=h14c3975_4 - yaml=0.1.7=had09818_2 - zlib=1.2.11=h7b6447c_3 prefix: /srv/conda daniel@f46b48213eb8:~$ conda env export name: notebook channels: - conda-forge/label/broken - conda-forge - defaults dependencies: - _libgcc_mutex=0.1=main - attrs=19.1.0=py_0 - backcall=0.1.0=py_0 - backports=1.0=py_2 - backports.functools_lru_cache=1.5=py_2 - backports.shutil_get_terminal_size=1.0.0=py27_2 - backports_abc=0.5=py_0 - blas=2.1=openblas - bleach=3.1.0=py_0 - bzip2=1.0.6=h14c3975_1002 - ca-certificates=2016.2.28=1 - certifi=2019.6.16=py27_0 - configparser=3.5.0=py27_1000 - cycler=0.10.0=py27_0 - dbus=1.13.6=h746ee38_0 - decorator=4.4.0=py27_1 - defusedxml=0.5.0=py_1 - entrypoints=0.3=py27_0 - enum34=1.1.6=py27_1 - expat=2.2.6=he6710b0_0 - fontconfig=2.13.0=h9420a91_0 - freetype=2.9.1=h8a8886c_1 - functools32=3.2.3.2=py27_1 - futures=3.2.0=py27_0 - glib=2.56.2=hd408876_0 - gst-plugins-base=1.14.0=hbbd80ab_1 - gstreamer=1.14.0=hb453b48_1 - icu=58.2=h9c2bf20_1 - ipaddress=1.0.22=py27_0 - ipykernel=4.8.2=py27_0 - ipython=5.8.0=py27_0 - ipython_genutils=0.2.0=py_1 - ipywidgets=7.4.2=py_0 - jedi=0.13.3=py27_0 - jinja2=2.10.1=py_0 - jpeg=9b=h024ee3a_2 - jsonschema=3.0.1=py27_0 - jupyter_client=5.2.4=py_3 - jupyter_core=4.4.0=py_0 - jupyterlab=0.33.11=py27_0 - jupyterlab_launcher=0.11.2=py27h28b3542_0 - kiwisolver=1.1.0=py27he6710b0_0 - libblas=3.8.0=1_openblas - libcblas=3.8.0=1_openblas - libffi=3.2.1=he1b5a44_1006 - libgcc-ng=8.2.0=hdf63c60_1 - libgfortran-ng=7.2.0=hdf63c60_3 - liblapack=3.8.0=1_openblas - liblapacke=3.8.0=1_openblas - libopenblas=0.3.6=h6e990d7_2 - libpng=1.6.37=hbc83047_0 - libsodium=1.0.16=h14c3975_1001 - libstdcxx-ng=8.2.0=hdf63c60_1 - libuuid=1.0.3=h1bed415_2 - libxcb=1.13=h1bed415_1 - libxml2=2.9.8=h422b904_1 - markupsafe=1.1.1=py27h7b6447c_0 - matplotlib=2.2.4=py27_0 - matplotlib-base=2.2.4=py27hfd891ef_0 - mistune=0.8.4=py27h7b6447c_0 - nbconvert=5.4.1=py_2 - nbformat=4.4.0=py_1 - ncurses=6.1=hf484d3e_1002 - notebook=5.7.4=py27_1000 - numpy=1.16.4=py27h99e49ec_0 - numpy-base=1.16.4=py27h2f8d375_0 - openblas=0.3.5=h9ac9557_1001 - openssl=1.1.1c=h7b6447c_1 - pandoc=2.7.2=0 - pandocfilters=1.4.2=py_1 - parso=0.4.0=py_0 - pathlib2=2.3.3=py27_0 - pcre=8.43=he6710b0_0 - pexpect=4.7.0=py27_0 - pickleshare=0.7.5=py27_0 - pip=19.1=py27_0 - prometheus_client=0.6.0=py_0 - prompt_toolkit=1.0.15=py27_0 - ptyprocess=0.6.0=py27_0 - pygments=2.4.2=py_0 - pyparsing=2.4.0=py_0 - pyqt=5.9.2=py27h05f1152_2 - pyrsistent=0.14.11=py27h7b6447c_0 - python=2.7.15=h721da81_1007 - python-dateutil=2.8.0=py27_0 - pytz=2019.1=py_0 - pyzmq=18.0.1=py27hc4ba49a_1 - qt=5.9.7=h5867ecd_1 - readline=7.0=hf8c457e_1001 - scandir=1.10.0=py27h7b6447c_0 - send2trash=1.5.0=py_0 - setuptools=41.0.1=py27_0 - simplegeneric=0.8.1=py27_2 - singledispatch=3.4.0.3=py27_0 - sip=4.19.8=py27hf484d3e_0 - six=1.12.0=py27_0 - sqlite=3.28.0=h8b20d00_0 - subprocess32=3.5.4=py27h7b6447c_0 - terminado=0.8.2=py27_0 - testpath=0.4.2=py_1001 - tk=8.6.9=h84994c4_1001 - tornado=5.1.1=py27h7b6447c_0 - traitlets=4.3.2=py27_0 - wcwidth=0.1.7=py27_0 - webencodings=0.5.1=py_1 - wheel=0.33.4=py27_0 - widgetsnbextension=3.4.2=py27_0 - wsgiref=0.1.2=py27_0 - xz=5.2.4=h14c3975_1001 - zeromq=4.3.1=hf484d3e_1000 - zlib=1.2.11=h14c3975_1004 prefix: /srv/conda/envs/notebook ```

I also changed the docs to not use -n root, so the default environment is exported, which is what users following those instructions probably want, right?

Happy to run tests (not done yet!) and potentially add a test if you think this goes in the right direction.