Issue with Python based tools if conda base environment is activated by default

[bebatut]

Hi,

I had an issue to test/serve tools which needs Python libraries (e.g. Scanpy, Anndata, Table compute on IUC):

https://gist.github.com/bebatut/f31bdb9342c5032e2c49a0580ec3eaa8

The tools were failing because the needed Python libraries were not found. But if we go inside the conda environment created by Galaxy (mulled-…) and start python, the libraries are there. Nothing interesting was in the log mentioning that the conda environment was not loaded.

After different tests (different versions of Planemo, conda, Galaxy), I found out that if I remove conda from the path and I do not have the conda base environment activated by default, I can test/serve the tools.

Any idea why? Thanks

Bérénice

[bernt-matthias]

I observed the same effect now on our production server. A users who had the conda init code in the .bashrc got the wrong python.

Problem seems related to the fact that conda actually does not prepend to PATH if base is activated but inserts the new elements just before base:

• activate base -> PATH is CONDA_BASE_BIN:...
• prepend something else to PATH, e.g. PATH="XYZ:$PATH" -> PATH is XYZ:CONDA_BASE_BIN:... (XYZ could be some python venv)
• activate another conda env -> PATH is XYZ:OTHER_CONDA_ENV_BIN:CONDA_BASE_BIN:...

seems that by default (need to verify this) Galaxy activates its virtualenv (which has its own python missing the required libraries).

[nsoranzo]

Following your order I correctly get as PATH: OTHER_CONDA_ENV_BIN:XYZ:CONDA_BASE_BIN:..

[bernt-matthias]

Maybe its because the odd conda init code in my bashrc:

# added by Miniconda3 4.5.12 installer
# >>> conda init >>>
# !! Contents within this block are managed by 'conda init' !!
__conda_setup="$(CONDA_REPORT_ERRORS=false '/home/berntm/miniconda3/bin/conda' shell.bash hook 2> /dev/null)"
if [ $? -eq 0 ]; then
    \eval "$__conda_setup"
else
    if [ -f "/home/berntm/miniconda3/etc/profile.d/conda.sh" ]; then
        . "/home/berntm/miniconda3/etc/profile.d/conda.sh"
        CONDA_CHANGEPS1=false conda activate base
    else
        \export PATH="/home/berntm/miniconda3/bin:$PATH"
    fi
fi
unset __conda_setup
# <<< conda init <<<

[bernt-matthias]

Which activates base by default on login.

[pcm32]

I have seen the same issue on planemo and on some of our galaxy instances. Yes, removing the >>> conda >>> ... <<< conda init <<< mentioned by @bernt-matthias from the service user running our instance and then restarting our processes (we use supervisor) fixed it.

What I noticed on inspection of the env for the job is that for some reason the galaxy venv binary path gets added at the beginning to the $PATH, making its python executable to appear before the one from the conda env for the tool.

[bernt-matthias]

Cool that you could verify this. On our system its worse, since jobs run as real user .. and if one activates base by default then Galaxy jobs using python of a conda env python won't work anymore.

I guess I need to continue with this: https://github.com/galaxyproject/galaxy/pull/9335