search

microsoft / vscode-python

Python extension for Visual Studio Code
https://aka.ms/pvsc-marketplace
MIT License
4.32k stars 1.18k forks source link

Conda activation is failing #4402

Closed MarekOzana closed 5 years ago

MarekOzana commented 5 years ago

Environment data

Expected behaviour

VSC opens interactive python using active py36 environment

Actual behaviour

  1. VSC is open with active ('py36': conda) environment
  2. Python: Show Python Interactive window
  3. Error message: Jupyter kernel cannot be started from 'Python 3.6.6 64-bit ('py36': conda)'. Using closest match Python 3.6.6 64-bit ('mro': conda) instead.

Steps to reproduce:

  1. Open VSC in empty directory
  2. Python: Select Interpreter
  3. select 'py36'
  4. Python: Show Python Interactive window

Logs

Relevant Part from Developer Tools > Console Log:

Python Extension: Python Extension (Error in getActivatedEnvironmentVariables, method:getActivatedEnvironmentVariables): 
Error: Command failed: 
C:/Users/marek.ozana.XCF/AppData/Local/Continuum/miniconda3/envs/py36/Scripts/activate && conda activate py36 && echo 'e8b39361-0157-4923-80e1-22d70d46dee6' && python C:/Users/marek.ozana.XCF/.vscode/extensions/ms-python.python-2019.1.0/pythonFiles/printEnvVariables.py
Could not find conda environment: py36
You can list all discoverable environments with `conda info --envs`.
    at ChildProcess.exithandler (child_process.js:303:12)
    at ChildProcess.emit (events.js:182:13)
    at maybeClose (internal/child_process.js:961:16)
    at Process.ChildProcess._handle.onexit (internal/child_process.js:248:5)
t.log @ console.ts:134

The problem seems to be that process is already in 'py36' environment and thus fails when trying to activate py36 again: C:/Users/marek.ozana.XCF/AppData/Local/Continuum/miniconda3/envs/py36/Scripts/activate && conda activate py36

MarekOzana commented 5 years ago

I re-installed the Don's extension "Jupyter 1.1.4" and it nicely opens Jupyter notebook in the correct environment. So the issue is only with the new interactive python. Jupyter Notebook log:

Starting Jupyter Notebook
jupyter notebook --no-browser --port=8888 --NotebookApp.allow_origin="*"
[I 10:24:22.506 NotebookApp] [nb_conda_kernels] enabled, 3 kernels found
[I 10:24:23.050 NotebookApp] The port 8888 is already in use, trying another port.
[I 10:24:23.051 NotebookApp] The port 8889 is already in use, trying another port.
[I 10:24:23.158 NotebookApp] [jupyter_nbextensions_configurator] enabled 0.4.1
[I 10:24:23.245 NotebookApp] JupyterLab extension loaded from C:\Users\marek.ozana.XCF\AppData\Local\Continuum\miniconda3\envs\py36\lib\site-packages\jupyterlab
[I 10:24:23.245 NotebookApp] JupyterLab application directory is C:\Users\marek.ozana.XCF\AppData\Local\Continuum\miniconda3\envs\py36\share\jupyter\lab
[I 10:24:23.257 NotebookApp] [nb_conda] enabled
[I 10:24:23.258 NotebookApp] Serving notebooks from local directory: d:\Projects\Python\Misc\BUG
[I 10:24:23.258 NotebookApp] The Jupyter Notebook is running at:
[I 10:24:23.258 NotebookApp] http://localhost:8890/?token=148d5e58990e52b07a6557afcc389e81a240e95a14d83f22
[I 10:24:23.258 NotebookApp] Use Control-C to stop this server and shut down all kernels (twice to skip confirmation).
[C 10:24:23.271 NotebookApp] 

    To access the notebook, open this file in a browser:
        file:///C:/Users/marek.ozana.XCF/AppData/Roaming/jupyter/runtime/nbserver-22232-open.html
    Or copy and paste one of these URLs:
        http://localhost:8890/?token=148d5e58990e52b07a6557afcc389e81a240e95a14d83f22
[I 10:24:25.815 NotebookApp] Kernel started: 08fa9738-4a01-4d45-9f57-6bf66ce9ba70
[I 10:24:27.038 NotebookApp] Adapting to protocol v5.1 for kernel 08fa9738-4a01-4d45-9f57-6bf66ce9ba70
MarekOzana commented 5 years ago

I noticed that Python: Create Terminal has the same issue with incorrect environment:

Microsoft Windows [Version 10.0.17763.316]
(c) 2018 Microsoft Corporation. All rights reserved.
D:\Projects\Python\EcoCharts\MarketUpdate>C:/Users/marek.ozana.XCF/AppData/Local/Continuum/miniconda3/envs/py36/Scripts/activate

(base) D:\Projects\Python\EcoCharts\MarketUpdate>conda activate py36
Could not find conda environment: py36
You can list all discoverable environments with `conda info --envs`.

It tries to activate 'py36' environment while already being in it. When listing environments I get:

(base) D:\Projects\Python\EcoCharts\MarketUpdate>conda info --envs
# conda environments:
#
                         C:\Users\marek.ozana.XCF\AppData\Local\Continuum\miniconda3
                         C:\Users\marek.ozana.XCF\AppData\Local\Continuum\miniconda3\envs\keras
                         C:\Users\marek.ozana.XCF\AppData\Local\Continuum\miniconda3\envs\mro
base                  *  C:\Users\marek.ozana.XCF\AppData\Local\Continuum\miniconda3\envs\py36
                         C:\Users\marek.ozana.XCF\AppData\Local\Continuum\miniconda3\envs\py37

So in summary VSC is in 'py36' environment but incorrectly calles it 'base'. However in this case the issue is not fatal since 'py36' is activated so one can continue working. In case of jupyter one cannot start 'py36'....

ronglums commented 5 years ago

Not data science specific. also happened in the terminal window.

benlindsay commented 5 years ago

Not data science specific. also happened in the terminal window.

I can confirm this. In an iterm2 window, when I type conda activate ads, I get a PATH variable containing paths in this order:

/Users/benlindsay/miniconda/envs/ads/bin
/Users/benlindsay/.rvm/gems/ruby-2.3.0/bin
/Users/benlindsay/.rvm/gems/ruby-2.3.0@global/bin
/Users/benlindsay/.rvm/rubies/ruby-2.3.0/bin
/Users/benlindsay/miniconda/condabin
/Users/benlindsay/.local/bin
/usr/local/bin
/Applications/R.app/Contents/MacOS
/Applications/RStudio.app/Contents/MacOS
/Applications/Ovito.app/Contents/MacOS
/usr/local/bin
/usr/bin
/bin
/usr/sbin
/sbin
/opt/X11/bin
/Library/TeX/texbin
/Users/benlindsay/.rvm/bin

In VSCode, if I have the python executable associated with the ads environment selected and start a new terminal, the resulting PATH contains these paths:

/Users/benlindsay/.rvm/gems/ruby-2.3.0/bin
/Users/benlindsay/.rvm/gems/ruby-2.3.0@global/bin
/Users/benlindsay/.rvm/rubies/ruby-2.3.0/bin
/Users/benlindsay/.local/bin
/usr/local/bin
/Applications/R.app/Contents/MacOS
/Applications/RStudio.app/Contents/MacOS
/Applications/Ovito.app/Contents/MacOS
/usr/local/bin
/usr/bin
/bin
/usr/sbin
/sbin
/opt/X11/bin
/Library/TeX/texbin
/Users/benlindsay/miniconda/envs/ads/bin
/Users/benlindsay/miniconda/condabin
/Users/benlindsay/.rvm/gems/ruby-2.3.0/bin
/Users/benlindsay/.rvm/gems/ruby-2.3.0@global/bin
/Users/benlindsay/.rvm/rubies/ruby-2.3.0/bin
/Users/benlindsay/.local/bin
/Applications/R.app/Contents/MacOS
/Applications/RStudio.app/Contents/MacOS
/Applications/Ovito.app/Contents/MacOS
/Users/benlindsay/.rvm/bin

So /Users/benlindsay/miniconda/envs/ads/bin was correctly added to PATH but it appears that my .zshrc was sourced again after doing so, which added duplicates of some paths in front of it, including /usr/bin, so that now the system python is the default interpreter instead of the conda environment interpreter. I'm guessing getting rid of that extra sourcing of the user's configuration file would fix the problem.

MarekOzana commented 5 years ago

I am not sure if you experience the same issue. I get this issue only on Windows. In summary: 1) create conda environment py36 2) select py36 in VSCode 3) creation of python terminal (or interactive python) fails because py36 environment cannot be activated inside of py36 environment.

On Windows I get this error (mind that it incorrectly uses ../envs/py36/../activate and not base activate):

.../Continuum/miniconda3/envs/py36/Scripts/activate
(base) C:\Test>conda activate py36
Could not find conda environment: py36
You can list all discoverable environments with `conda info --envs`

On Linux the activation correctly uses the base conda envirnment and works nicely:

source /home/.../miniconda3/bin/activate
conda activate py36

I tried on several different Windows computers using clean miniconda install and I am getting allways teh same issue. It stops me from using interactive python on Windows. The only workaround right now is to use the deprecated Don's jupyter extension, but unfortunately this does not work with python 3.7. Thank you for your time and help fixing this bug.

benlindsay commented 5 years ago

Ah, OK, maybe I'll start a separate issue for my problem then.

DonJayamanne commented 5 years ago

@MarekOzana

MarekOzana commented 5 years ago

I opened VSCOde, selected py36 interpreter and run command: Python: Show Interactive Window See attached

The problem seems to be that the extension tries to activate py36 environment while already being in the py36 environment, and thus cannot find it.

Thank you for your time Don.

MarekOzana commented 5 years ago

Don, does my comment above give you all the info you need?

DonJayamanne commented 5 years ago

@MarekOzana

  1. VSC is open with active ('py36': conda) environment

How did you manage to do this? Did you activate py36 in the terminal and then launch VSC from within that terminal?

MarekOzana commented 5 years ago

@DonJayamanne

I never start VSCode from a terminal with an already activated conda environment. conda and python are not even on path. Correct way of activating environment is:

c:\Users\marek.ozana.XCF\AppData\Local\Continuum\miniconda3\Scripts\activate
(base) C:\>conda activate py36
(py36) C:\>

While it seems that VScode-python extension is activating inside of environment and then tries to activate it again:

C:/Users/marek.ozana.XCF/AppData/Local/Continuum/miniconda3/envs/py36/Scripts/activate
(base) C:\>conda activate py36
Could not find conda environment: py36
You can list all discoverable environments with `conda info --envs`.
DonJayamanne commented 5 years ago

Expected approach for activation:

c:\Users\marek.ozana.XCF\AppData\Local\Continuum\miniconda3\Scripts\activate
(base) C:\>conda activate py36
(py36) C:\>

What we're doing is as follows:

ghost commented 5 years ago

My problem is most likely related.

I’ve selected the Python interpreter in settings.json: "python.pythonPath": "C:\\Anaconda3\\envs\\copernicus\\python.exe"

A new terminal opens fine in the correct environment, see below.

Microsoft Windows [Version 6.1.7601]
Copyright (c) 2009 Microsoft Corporation.  All rights reserved.

C:\DATA\Copernicus
> C:/Anaconda3/Scripts/activate

(base) C:\DATA\Copernicus
> conda activate copernicus

(copernicus) C:\DATA\Copernicus
>

But, Python Interactive window is still using the base environment, C:\\Anaconda3\\python.exe, see below.

Jupyter Server URI: http://localhost:8888/?token=...
Python Version:
3.6.5 | packaged by conda-forge | (default, Apr  6 2018, 16:13:55) [MSC v.1900 64 bit (AMD64)]
C:\\Anaconda3\\python.exe
Jupyter Notebook Version: (5, 4, 1)

I would expect the Interactive window using the same interpreter as the terminal and as given in settings.json.

bushidocodes commented 5 years ago

I am experiencing this same issue using the Windows Subsystem for Linux. My state seems to be identical to @ererkka.

With a directory containing a python 3 Hello World,

I created a new Conda environment using

conda create -n python-masterclass python=3.7
conda install pylint
conda install ipykernel

I closed VS Code and reopened it so that the conda environment showed up in the toolbar I selected the python-masterclass interpreter and VSCode automatically set the following in ./.vscode/settings.json:

{
    "python.linting.pylintEnabled": true,
    "python.pythonPath": "/home/sean/miniconda/envs/python-masterclass/bin/python"
}

The terminal starts cleanly and I was able to shift-enter and run code in the terminal without a problem.

I got the popup prompt "Do you want to use the new Interactive Editor when you shift-enter?" I clicked yes.

I get the following nag, showing me that I'm not running in the interpreter I expect: image

It's unclear how to disable the new interactive editor or revert back to running in the terminal.

ghost commented 5 years ago

I think you can disable the Interactive Editor from the settings: python.dataScience.enabled is the key.

DonJayamanne commented 5 years ago

@bushidocodes Please file a separate issue for what you're experiencing, it is not related to activation of conda environments. As the message indciates the problem is not being able to locate jupyter kerner in your environment.

MarekOzana commented 5 years ago

Any progress on this one? I had a workaround which worked until the latest python extension update: 1) Activate base conda environment in terminal, 2) start code from this terminal 3) open Interactive window. Unfortunately even this stopped working after the last python extension update. Pity, such a great editor, if just jupyter interactive window worked....

MarekOzana commented 5 years ago

I found out that the issue happens only if conda-build package is installed in the environment. I created two clean environments.

Then I tried to start python interactive window in VSCode in each of the environment. It succeeded in the first environment (without conda-build package: ok_env): ok_env and it failed in the second environment (with conda-build package installed: bug_env): bug_env

So in summary, there is something with conda-build package being installed in the environment which is messing up VSCode activation.

DonJayamanne commented 5 years ago

Are you able to activate the conda environment from the terminal (manually). I.e. does conda activate bug_env work in the terminal?

MarekOzana commented 5 years ago

Hi Don,Yes activation of the environment form console works fine.

(base) C:\>conda activate bug_env
(bug_env) C:\>

Comment whether or not this issue is another bug

I think this issue is the same as reported above:

  1. VSCode is first calling activate inside of bug_env environment C:/Users/mare.../envs/bug_env/Scripts/activate
  2. and then tries to activate the environment again conda activate bug_env VSCOde-bug

See the full log file in the attachment: VSCode-bug_env_log.log

Also if one tries to start python terminal from bug_env one gets familiar error:

D:\>C:/Users/mare.../envs/bug_env/Scripts/activate
(base) D:\>conda activate bug_env
Could not find conda environment: bug_env
You can list all discoverable environments with `conda info --envs`.

and when listing conda environments in this terminal window one gets:

(base) D:\>conda info --envs
# conda environments:
#
                         C:\Users\marek.ozana.XCF\AppData\Local\Continuum\miniconda3
base                  *  C:\Users\marek.ozana.XCF\AppData\Local\Continuum\miniconda3\envs\bug_env
                         C:\Users\marek.ozana.XCF\AppData\Local\Continuum\miniconda3\envs\live

so bug_env is activated and fails to activate itself again.

All of the above works fine in ok_env which differs just by not having conda-build: VSCode-ok_env

xhochy commented 5 years ago

I ran into the same problem and tracked down the source of the issue to https://github.com/Microsoft/vscode-python/blob/8cd0996e4fbb50f5aa4ca2d05b320a370ae121a6/src/client/interpreter/locators/services/condaService.ts#L254-L262

Here we try to activate the environment using the conda installation that is installed in the environment. I cannot think of a situation where this would work. One must activate the conda that created the environment hence this cannot be the conda installed in that environment as we would otherwise have a chicken-egg situation.

@rchiodo Maybe you can add more context why these two blocks exist as you wrote the environment activation code initially in https://github.com/Microsoft/vscode-python/pull/3506 ?

rchiodo commented 5 years ago

Sorry I must be missing something. That line of code is just looking in the Scripts folder for the conda executable next to the python that was found. Why is this causing a problem?

The way this stuff was supposed to work was we call conda activate for the base and then inside the base, call the conda activate 'env-name'. It looks like we don't find the base. I'm guessing we're using the python.exe path and assuming it's for the 'base' and that there isn't one for the sub environment.

For example, here's what happens on my own machine:

Python Extension: Activating Environment to capture Environment variables, C:/Users/rchiodo/AppData/Local/Continuum/anaconda3/Scripts/activate && conda activate jupyter && echo 'e8b39361-0157-4923-80e1-22d70d46dee6' && python f:/vscode-python-1/pythonFiles/printEnvVariables.py

C:/Users/rchiodo/AppData/Local/Continuum/anaconda3/Scripts/activate is where the base conda was installed. It's next to where the python.exe is found.

The 'jupyter' environment is the secondary environment it's activating.

It looks like for @MarekOzana, the first activate is the wrong one. We must be using the python installed in the environment instead.

xhochy commented 5 years ago

@rchiodo The problem is only occuring when you have conda installed in the non-base environment.

The following steps show the issue:

(base) root@e1176206961b:/home# 



Removing the code lines I mentioned earlier resolves the problem.
rchiodo commented 5 years ago

Ah that's because it doesn't find an activate so falls back to the default location.

Is it normal to install conda in a test environment? Is that so you can have a different conda?

xhochy commented 5 years ago

It is not normal but still happening from time to time. You can use conda programmatically to inspect your environment and also use this to build packages.

In the case of conda environments, I would never call the activate scripts but only ever call the default location. activate should only be called in virtualenv situations. The above linked code is though only executed when we're already sure that we're dealing with a conda environment.

mingwandroid commented 5 years ago

Is it normal to install conda in a test environment? Is that so you can have a different conda?

Now-a-days, conda is only ever installed in the base env. We don't even put hooks to it into other envs anymore. Instead, the base env's condabin directory remains on PATH always.

xhochy commented 5 years ago

Is it normal to install conda in a test environment? Is that so you can have a different conda?

Now-a-days, conda is only ever installed in the base env. We don't even put hooks to it into other envs anymore. Instead, the base env's condabin directory remains on PATH always.

Maybe to add my usecase why I have conda installed in a conda environment itself: I have code that uses conda itself to get information about an environment and also creates a set of environments out of this environment.

mingwandroid commented 5 years ago

Maybe to add my usecase why I have conda installed in a conda environment itself

How have you done this? Conda prevents it!

MarekOzana commented 5 years ago

The bug happens when I install conda-build package (used to build and develop packages locally) inside an environment. The steps are describe in my post above: https://github.com/microsoft/vscode-python/issues/4402#issuecomment-486310389

conda create -n bug_env python jupyter conda-build

xhochy commented 5 years ago

Maybe to add my usecase why I have conda installed in a conda environment itself

How have you done this? Conda prevents it!

Just like with any other package:

% conda create -p $(pwd)/test python=3.7 conda
Collecting package metadata: done
Solving environment: done

## Package Plan ##

  environment location: /Users/uwe/test

  added / updated specs:
    - conda
    - python=3.7
…
% conda activate /Users/uwe/test
% ls test/lib/python3.7/site-packages/conda
__init__.py     __pycache__     activate.py     base            common          core            exports.py      history.py      instructions.py misc.py         plan.py         shell
__main__.py     _vendor         api.py          cli             compat.py       exceptions.py   gateways        install.py      lock.py         models          resolve.py      utils.py
% ls test/bin/conda
test/bin/conda
% ls test/bin/activate
test/bin/activate

Also it should not prevent it as I want to use the Python API of the package: https://docs.conda.io/projects/conda/en/latest/api/