search

pypa / virtualenv

Virtual Python Environment builder
https://virtualenv.pypa.io
MIT License
4.84k stars 1.03k forks source link

"source ./miniconda-py-3.9.20/bin/activate" does not give access to conda when running "python -m conda" #2811

Open 0x303 opened 13 hours ago

0x303 commented 13 hours ago

Goal

I want to be able to activate my miniconda virtual environment and then run conda and jupyter like this:

# Activates miniconda virtual environment

$ python -m conda

$ python -m jupyter

I want to do this because when my virtual environment is activated I know that I am running my specified version of python, and using the verbose python -m <module_name> syntax ensures I am running the subcommand of my specified python version, and not a globally installed equivalent.

The issue is, I can't seem to get this to work reliably. After installing my specified miniconda python version with pyenv, I then create a virtual environment using the virtualenv command. This gives me access to do python -m jupyter but not python -m conda.

If I install pyenv-virtualenv as a pyenv plugin and then do pyenv install miniconda3-3.9-24.9.2-0 and then run pyenv activate <python_version_name>, like in pyenv activate miniconda3-3.9-24.9.2-0, then I will be able to run python -m conda but not python -m jupyter.

As an edge case, I noticed if I instead use pyenv-virtualenv to create a new miniconda environment (instead of using the default one) by giving it a custom name with the pyenv <python_version> <environment_name> command and then activate the environment with pyenv activate <environment_name> I will not be able to use python -m conda nor python -m jupyter.

I am not sure what I am doing wrong... I just want to be able to run both subcommands in a miniconda virtual environment using the verbose python -m <module_name> syntax.

0x303 commented 13 hours ago

Super Detailed Background

I am using pyenv to install independent version of python. I ran the following commands to setup and install a virtual environment.

$ pyenv install miniconda3-3.9-24.9.2-0

# Installs miniconda3-3.9-24.9.2-0 to ~/.pyenv/versions/miniconda3-3.9-24.9.2-0

$ mkdir ~/Documents/VirtualEnvironments/miniconda3-3.9-24.9.2-0

$ cd  ~/Documents/VirtualEnvironments/miniconda3-3.9-24.9.2-0

$ virtualenv -p ~/.pyenv/versions/miniconda3-3.9-24.9.2-0/bin/python ./

Now when I source this environment I SHOULD be using Python 3.9.20 with all of the miniconda packages (including the conda binary). 

$ source ~/Documents/VirtualEnvironments/miniconda3-3.9-24.9.2-0/bin/activate

$ python --version
Python 3.9.20

Trying to run python -m conda will fail, but running a different binary that comes with miniconda like jupyter will work fine:

# Using 🐍 Python v3.9.20(miniconda3-3.9-24.9.2-0)
$ python -m conda --version
/Users/me/Documents/VirtualEnvironments/miniconda3-3.9-24.9.2-0/bin/python: No module named conda

$ python -m jupyter --version                                                
Selected Jupyter core packages...
IPython          : 8.18.1
ipykernel        : 6.29.5
ipywidgets       : 8.1.5
jupyter_client   : 8.6.3
jupyter_core     : 5.7.2
jupyter_server   : 2.14.2
jupyterlab       : 4.2.6
nbclient         : 0.10.0
nbconvert        : 7.16.4
nbformat         : 5.10.4
notebook         : 7.2.2
qtconsole        : 5.6.1
traitlets        : 5.14.3

I am guessing this is due to the way the PATH environment variable is being set by virtualenv.

Here is my PATH and some other pertinent information for debugging purposes:

# BEFORE sourcing the Virtual Environment:

$ echo $PATH
/Users/me/.pyenv/plugins/pyenv-virtualenv/shims:/Users/me/.pyenv/shims:/opt/homebrew/bin:/opt/homebrew/sbin:/usr/local/bin:/usr/bin:/bin:/usr/sbin:/sbin:/Users/me/go/bin

$ which jupyter 
/opt/homebrew/bin/jupyter

# AFTER sourcing the Virtual Environment

$ echo $PATH
/Users/me/Documents/VirtualEnvironments/miniconda3-3.9-24.9.2-0/bin:/Users/me/.pyenv/plugins/pyenv-virtualenv/shims:/Users/me/.pyenv/shims:/opt/homebrew/bin:/opt/homebrew/sbin:/usr/local/bin:/usr/bin:/bin:/usr/sbin:/sbin:/Users/me/go/bin

$ which jupyter 
/Users/me/Documents/VirtualEnvironments/miniconda3-3.9-24.9.2-0/bin/jupyter

However for whatever reason, the conda binary is not in the path because it is not added to the ~/Documents/VirtualEnvironments/miniconda3-3.9-24.9.2-0/bin directory.

Work Around that breaks Jupyter

If I instead install pyenv-virtualenv as a plugin to pyenv by doing $ git clone https://github.com/pyenv/pyenv-virtualenv.git $(pyenv root)/plugins/pyenv-virtualenv, then I can use the command pyenv activate <virtual_environment> instead of have to do source <virtual_environment>/bin/activate.

This will let me use the conda command fine, but it will not let you use jupyter and that will work fine:

$ pyenv activate miniconda3-3.9-24.9.2-0

# Now using miniconda3-3.9-24.9.2-0

$ python -m conda --version             
conda 24.9.2

$ python -m jupyter --version           
/Users/hankfaust/.pyenv/versions/miniconda3-3.9-24.9.2-0/bin/python: No module named jupyter

# EXTRA INFO ON CONDA BINARY LOCATION and PATH
$ which conda      
conda () {
        \local cmd="${1-__missing__}"
        case "$cmd" in
                (activate | deactivate) __conda_activate "$@" ;;
                (install | update | upgrade | remove | uninstall) __conda_exe "$@" || \return
                        __conda_reactivate ;;
                (*) __conda_exe "$@" ;;
        esac
}

$ echo $CONDA_EXE            
/Users/me/.pyenv/versions/miniconda3-3.9-24.9.2-0/bin/conda

$ echo $PATH
/Users/me/.pyenv/versions/miniconda3-3.9-24.9.2-0/condabin:/Users/me/.pyenv/plugins/pyenv-virtualenv/shims:/Users/me/.pyenv/shims:/opt/homebrew/bin:/opt/homebrew/sbin:/usr/local/bin:/usr/bin:/bin:/usr/sbin:/sbin:/Users/me/go/bin

Observation

By observing the way the $PATH variable changes between the different ways to activate the virtual environments I noticed this: