Python3 interpreter is not working in the archived environment

[xdriver]

Bug description

I have followed the manual installation guide (which was actually not too hard to understand) and tried to get JupyterHub up and running within a yarn-based archived environment. So far, when starting the JupytherHub environment, everything seems normal. Only when a notebook is created, the python3 interpreter fails.

It reports a kernel error and the last lines show:

... File "/hadoop/yarn/local/usercache/{jupyter_user}/appcache/{yarn_application_id}/environment/lib/python3.8/subprocess.py", line 1702, in _execute_child raise child_exception_type(errno_num, err_msg, err_filename) FileNotFoundError: [Errno 2] No such file or directory: '/home/conda/feedstock_root/build_artifacts/ipykernel_1595446881062/_h_env_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_p/bin/python'

If I execute some commands within the terminal, I get the following information:

sh-4.2$ jupyter kernelspec list /hadoop/yarn/local/usercache/{jupyter_user}/appcache/{yarn_application_id}/{yarn_containiner_id}/environment/lib/python3.8/site-packages/IPython/paths.py:66: UserWarning: IPython parent '/home' is not a writable location, using a temp directory. warn("IPython parent '{0}' is not a writable location," Available kernels: python3 /hadoop/yarn/local/usercache/{jupyter_user}/appcache/{yarn_application_id}/{yarn_containiner_id}/environment/share/jupyter/kernels/python3

The directory and file mentioned for python3 exists and can be accessed.

Actual behaviour

How to reproduce

I followed the manual installation guide. Trying to overcome the obstacle by myself, I just extended one command for creating the conda environment as following:

conda create -c conda-forge -y -p /opt/jupyterhub/envs/example python=3.8 ipykernel

Also with the standard command as provided in the installation guide (conda create -c conda-forge -y -p /opt/jupyterhub/envs/example), without specifying the python environment and ipykernel the results are the same.

The setup is using a dummy authentication. HTTPS is enabled. The effect is the same independent from the user, who has logged in. HDFS support for storing the notebooks in hadoop's hdfs is enabled and properly working.

Your personal set up

• OS:
• Version: Linux CentOS7 (latest patch level)
• Configuration:

sh-4.2$ jupyterhub --version /hadoop/yarn/local/usercache/{jupyter_user}/appcache/{yarn_application_id}/{yarn_containiner_id}/environment/bin/jupyterhub: line 2: /home/conda/feedstock_root/build_artifacts/jupyterhub-feedstock_1597233557917/_h_env_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_/bin/python: No such file or directory

No harddsik encryption (this was mentioned as a root cause in similar issues that I have investigated related to this problem).

[langustav]

Hi, any update regarding this topic? Thank you. Michal

[rufytn]

hi, @langustav were you able to solve?