Error when switching to conda tab, referencing /tmp/jupyter_conda_packages.json

jrderuiter commented 4 years ago

Description

When trying to create a docker image for JupyterHub containing jupyter_conda, I'm getting the following error in the terminal:

Fail to load available packages from cache [Errno 2] No such file or directory: '/tmp/jupyter_conda_packages.json

After a few moments, this error does seem to resolve as /tmp/jupyter_conda_packages.json is created by the notebook server.

I have two questions regarding this behaviour:

Why is the file only showing up later? Is there a way to get this file loaded before? (Currently it is giving an error in the UI as well, together with the terminal error, which is worrying our users needlessly.)
The jupyter_conda_packages.json file is being writting to /tmp/jupyter_conda_packages.json, and is being owned by the user who first spawned his/her notebook. Will this work in a multi-user setup?

Reproduce

I'm creating a docker image using:

ARG PYTHON_VERSION=3.6
FROM godatadriven/miniconda:${PYTHON_VERSION}

ARG BUILD_DATE
ARG JH_VERSION

LABEL org.label-schema.name="JupyterHub $JH_VERSION" \
      org.label-schema.build-date=$BUILD_DATE \
      org.label-schema.version=$JH_VERSION

RUN set -x && \
    if [ -n "$JH_VERSION" ]; then\
        conda install -y nb_conda_kernels jupyter_conda jupyterhub==$JH_VERSION jupyter_client 'tornado<6' -c conda-forge;\
    else\
        conda install -y nb_conda_kernels jupyter_conda jupyterhub jupyter_client 'tornado<6' -c conda-forge;\
    fi && \
    python -m nb_conda_kernels.install --enable --prefix=/opt/miniconda3 && \
    conda clean -tipsy && \
    apt-get update && \
    apt-get install -y git build-essential nano vim less procps apt-transport-https ca-certificates --no-install-recommends && \
    apt-get clean

RUN useradd -ms /bin/bash -p "$(openssl passwd -1 joyvan)" joyvan

EXPOSE 8000
ENTRYPOINT ["jupyterhub"]

I then start the server using:

docker run --rm -p 8000:8000 jupyterhub:latest --ip 0.0.0.0

and login using user joyvan with password joyvan.

Expected behavior

For the first question, I would expect not to get this error + get packages loaded in the left bottom window of the conda tab on the first view.
For the second, I would expect the json file to be written in a user-specific location rather than a global location that is the same for all users.

Context

Python package version: 3.1.1
Extension version: n/a
General information:

Conda info:

     active environment : None
       user config file : /home/joyvan/.condarc
 populated config files : 
          conda version : 4.7.12
    conda-build version : not installed
         python version : 3.6.9.final.0
       virtual packages : 
       base environment : /opt/miniconda3  (read only)
           channel URLs : https://repo.anaconda.com/pkgs/main/linux-64
                          https://repo.anaconda.com/pkgs/main/noarch
                          https://repo.anaconda.com/pkgs/r/linux-64
                          https://repo.anaconda.com/pkgs/r/noarch
          package cache : /opt/miniconda3/pkgs
                          /home/joyvan/.conda/pkgs
       envs directories : /home/joyvan/.conda/envs
                          /opt/miniconda3/envs
               platform : linux-64
             user-agent : conda/4.7.12 requests/2.22.0 CPython/3.6.9 Linux/4.9.184-linuxkit debian/9 glibc/2.24
                UID:GID : 1000:1000
             netrc file : None
           offline mode : False

fcollonval commented 4 years ago

Dear @jrderuiter thanks for the detailed issue.

Why is the file only showing up later?

Retrieving the list of available packages is slow - especially because the package description is not available in the package info. So the purpose of the file /tmp/jupyter_conda_packages.json is to serve as cache file for the available package list:

https://github.com/fcollonval/jupyter_conda/blob/7856fa90585d5a00cb80d6172974dd5b05d8213e/jupyter_conda/handlers.py#L345-L355

Is there a way to get this file loaded before?

It could be taken from a previously run container (or personal installation) to init the cache file.

Retrieving the available package list with their description could definitely be improved. If you get some idea, I will be happy to discuss them.

Currently it is giving an error in the UI as well, together with the terminal error, which is worrying our users needlessly.

This one is not related to the warning, you are seeing in the terminal. Could you look at the web browser console to see the error being reported to the user in the frontend?

The jupyter_conda_packages.json file is being writting to /tmp/jupyter_conda_packages.json, and is being owned by the user who first spawned his/her notebook. Will this work in a multi-user setup?

Thanks for pointing this. You are totally right. And this should be corrected. As that file is doing caching, it would make sense to get it readable/writable by anyone (possible fix using os.open

If you want, I will be more than happy to include contribution to improve package list retrieval.

fcollonval commented 4 years ago

Closing as no further information was provided about the reported error in the UI and the error could not be reproduced.

mamba-org / gator