search

microsoft / vscode-python

Python extension for Visual Studio Code
https://aka.ms/pvsc-marketplace
MIT License
4.34k stars 1.19k forks source link

Modify `getActiveEnvironmentPath()` API when `resource` is passed as `undefined` in a multiroot workspace #21256

Open rdbisme opened 1 year ago

rdbisme commented 1 year ago

Type: Bug

Behaviour

Black server doesn't start because it requires an interpreter configured, even if "black-formatter.importStrategy": "fromEnvironment".

Expected vs. Actual

It should use the environment python as interpreter if fromEnvironment is setup and not require the additional setting.

Steps to reproduce:

  1. Open a python file
  2. Black doesn't start because of missing interpreter

Diagnostic data

Output for Python in the Output panel (View→Output, change the drop-down the upper-right of the Output panel to Python)

``` XXX ```

User Settings ```json { "workbench.tree.indent": 15, "terminal.integrated.profiles.windows": { "Git Bash (Scoop)": { "path": "C:\\Users\\xxxx\\scoop\\apps\\git\\current\\bin\\bash.exe", "args": ["--login"] }, "UCRT64": { "path": "C:\\Users\\xxxx\\scoop\\apps\\msys2\\current\\usr\\bin\\bash.exe", "args": ["--login", "-i"], "env": { "MSYSTEM": "UCRT64", "CHERE_INVOKING": "1", "MSYS2_PATH_TYPE": "inherit" } }, "MINGW64": { "path": "C:\\Users\\xxxx\\scoop\\apps\\msys2\\current\\usr\\bin\\bash.exe", "args": ["--login", "-i"], "env": { "MSYSTEM": "MINGW64", "CHERE_INVOKING": "1", "MSYS2_PATH_TYPE": "inherit" } } }, "workbench.colorTheme": "Solarized Dark+", "python.defaultInterpreterPath": "C:\\Users\\xxxx\\scoop\\shims\\python3", "python.venvPath": "C:\\Users\\xxxx\\AppData\\Local\\pypoetry\\Cache\\virtualenvs", "python.formatting.provider": "none", "editor.formatOnSave": true, "isort.args": ["--profile", "black"], "[python]": { "editor.defaultFormatter": "ms-python.black-formatter", "editor.codeActionsOnSave": { "source.organizeImports": true }, "editor.formatOnType": true }, "[terraform]": { "editor.formatOnSave": true, "editor.defaultFormatter": "hashicorp.terraform", "editor.tabSize": 2 // optionally }, "python.analysis.logLevel": "Error", "terminal.integrated.defaultProfile.windows": "UCRT64", "editor.rulers": [80, 120], "vim.useSystemClipboard": true, "vim.leader": ",", "vim.foldfix": true, "vim.normalModeKeyBindings": [ { "before": ["", "r"], "commands": ["editor.action.rename"] }, { "before": ["", "R"], "commands": ["editor.action.refactor"] }, { "before": ["", "d"], "commands": ["editor.action.revealDefinition"] }, { "before": ["", "t"], "commands": ["editor.action.goToTypeDefinition"] } ], "diffEditor.ignoreTrimWhitespace": false, "editor.fontFamily": "MesloLGM NF, Consolas, 'Courier New', monospace, 'Segoe UI Emoji'", "rust-client.rustupPath": "C:\\Users\\xxxx\\scoop\\persist\\rustup\\.cargo\\bin\\rustup.exe", "security.workspace.trust.untrustedFiles": "open", "jupyter.askForKernelRestart": false, "terminal.integrated.fontFamily": "MesloLGM NF", "terminal.integrated.fontWeightBold": "bold", "editor.fontSize": 13, "explorer.confirmDelete": false, "[json]": { "editor.defaultFormatter": "esbenp.prettier-vscode" }, "vim.neovimConfigPath": "C:\\Users\\xxxx\\scoop\\shims\\nvim", "vim.enableNeovim": true, "remote.SSH.path": "C:\\Users\\xxxx\\scoop\\apps\\msys2\\current\\usr\\bin\\ssh.exe", "redhat.telemetry.enabled": true, "workbench.editorAssociations": { "git-rebase-todo": "default" }, "[javascript]": { "editor.defaultFormatter": "esbenp.prettier-vscode" }, "[yaml]": { "editor.defaultFormatter": "redhat.vscode-yaml" }, "http.proxyStrictSSL": true, "[jsonc]": { "editor.defaultFormatter": "esbenp.prettier-vscode" }, "black-formatter.importStrategy": "fromEnvironment", "black-formatter.interpreter": [ "C:\\Users\\xxxx\\scoop\\shims\\python.exe" ], "git.openRepositoryInParentFolders": "never", "window.zoomLevel": -1, "[nginx]": { "editor.defaultFormatter": "ahmadalli.vscode-nginx-conf" } } ```

``` Multiroot scenario, following user settings may not apply: venvPath: "" languageServer: "Pylance" formatting • provider: "none" ```

Extension version: 2023.2.0 VS Code version: Code 1.78.1 (6a995c4f4cc2ced6e3237749973982e751cb0bf9, 2023-05-04T09:48:08.683Z) OS version: Windows_NT x64 10.0.19044 Modes: Sandboxed: Yes

System Info |Item|Value| |---|---| |CPUs|Intel(R) Core(TM) i7-8700 CPU @ 3.20GHz (12 x 3192)| |GPU Status|2d_canvas: enabled
canvas_oop_rasterization: disabled_off
direct_rendering_display_compositor: disabled_off_ok
gpu_compositing: enabled
multiple_raster_threads: enabled_on
opengl: enabled_on
rasterization: enabled
raw_draw: disabled_off_ok
video_decode: enabled
video_encode: enabled
vulkan: disabled_off
webgl: enabled
webgl2: enabled
webgpu: enabled| |Load (avg)|undefined| |Memory (System)|31.85GB (15.06GB free)| |Process Argv|--enable-proposed-api genuitecllc.codetogether --crash-reporter-id ed01675f-67d1-4c85-8363-79de19e446c3| |Screen Reader|no| |VM|0%|
A/B Experiments ``` vsliv368:30146709 vsreu685:30147344 python383cf:30185419 vspor879:30202332 vspor708:30202333 vspor363:30204092 vswsl492:30256859 vslsvsres303:30308271 vserr242:30382549 pythontb:30283811 vsjup518:30340749 pythonptprofiler:30281270 vsdfh931:30280409 vshan820:30294714 vstes263:30335439 vscoreces:30445986 pythondataviewer:30285071 vscod805cf:30301675 binariesv615:30325510 bridge0708:30335490 bridge0723:30353136 cmake_vspar411:30581797 vsaa593cf:30376535 pythonvs932:30410667 cppdebug:30492333 vsclangdc:30486549 c4g48928:30535728 dsvsc012:30540252 pynewext54:30695312 azure-dev_surveyone:30548225 nodejswelcome1cf:30587006 3biah626:30602489 pyind779:30671433 f6dab269:30613381 pythonsymbol12:30671437 vsctsb:30705553 azdwalk:30687957 pythonms35:30701012 pythonclientmvcf:30708686 ```
rdbisme commented 1 year ago

If I specify the interpreter the server starts, but then it doesn't use the environment

karthiknadig commented 1 year ago

@rdbisme Please provide logs from Output > Black Formatter. Also, provide path to the environment where you have installed black.

rdbisme commented 1 year ago 
2023-05-10 18:13:31.375 [info] Name: Black Formatter
2023-05-10 18:13:31.375 [info] Module: black
2023-05-10 18:13:31.376 [info] Python extension loading
2023-05-10 18:13:31.376 [info] Waiting for interpreter from python extension.
2023-05-10 18:13:39.347 [info] Python extension loaded
2023-05-10 18:13:39.408 [error] Python interpreter missing:
[Option 1] Select python interpreter using the ms-python.python.
[Option 2] Set an interpreter using "black-formatter.interpreter" setting.
Please use Python 3.7 or greater.
2023-05-10 18:13:57.057 [info] Formatting requested before server has started.

I have a multiproject workspace, so I have a virtualenvironment selected per project.

(monitoring-WTCPidQC-py3.10)
xxx@xxxxUCRT64 ~/git/***-monitoring (delay)
$ which black
/c/Users/xxx/AppData/Local/pypoetry/Cache/virtualenvs/monitoring-WTCPidQC-py3.10/Scripts/black

monitoring-WTCPidQC-py3.10 is the virtualenv configured for the project on which I have triggered the format document.

karthiknadig commented 1 year ago

@rdbisme Can you share your *.code-workspace?

rdbisme commented 1 year ago

A sanitized version: 

{
  "folders": [
    {
      "name": "****_prod",
      "path": "****/src/****_prod"
    },
    {
      "name": "****",
      "path": "****"
    },
    {
      "name": "qp",
      "path": "****-deploy/src/qp"
    },
    {
      "name": "****-strats-ci",
      "path": "****-strats-ci"
    },
    {
      "name": "xxxy0",
      "path": "xxxy0"
    },
    {
      "name": "****-ci",
      "path": "****-ci"
    },
    {
      "name": "backfill-ci",
      "path": "****-deploy/src/backfill-ci"
    },
    {
      "name": "internal-docs",
      "path": "****-deploy/src/internal-docs"
    },
    {
      "path": "****-api"
    },
    {
      "path": "****-monitoring"
    },
    {
      "path": "****-deploy/src/cmd-batch"
    },
    {
      "path": "****-ui"
    }
  ],
  "settings": {
    "yaml.schemas": {
      "https://gitlab.com/gitlab-org/gitlab/-/raw/master/app/assets/javascripts/editor/schema/ci.json": "file:///c%3A/Users/xxxxx/git/****-strats-ci/template-research-ci.yml"
    }
  },
  "launch": {
    "version": "0.2.0",
    "configurations": [
      {
        "name": "Python: Debug Tests",
        "type": "python",
        "request": "launch",
        "program": "${file}",
        "purpose": ["debug-test"],
        "console": "integratedTerminal",
        "justMyCode": false,
        "env": { "PYTEST_ADDOPTS": "--no-cov -v" }
      }
    ],
    "compounds": []
  }
}
karthiknadig commented 1 year ago

@rdbisme Can you share the logs from the python extension? Also, when you select interpreter for your multi root scenario did you select for the entire workspace or per root?

rdbisme commented 1 year ago

@rdbisme Can you share the logs from the python extension? Also, when you select interpreter for your multi root scenario did you select for the entire workspace or per root?

I selected per root. And Intellisense and completions work as expected. Do you still need the logs?

karthiknadig commented 1 year ago

@rdbisme Can you select one for the entire workspace as well? and see if that helps.

Another option is to add the interpreter here, (this is a temporary solution):

  "settings": {
    "yaml.schemas": {
      "https://gitlab.com/gitlab-org/gitlab/-/raw/master/app/assets/javascripts/editor/schema/ci.json": "file:///c%3A/Users/xxxxx/git/****-strats-ci/template-research-ci.yml"
    },
    "black-formatter.interpreter": ["c:/Users/xxx/AppData/Local/pypoetry/Cache/virtualenvs/monitoring-WTCPidQC-py3.10/Scripts/python"]
  },

This is scenario that is not currently handled well. I will need to investigate what we can do to provide a better fix for this in these cases.

rdbisme commented 1 year ago

@rdbisme Can you select one for the entire workspace as well? and see if that helps.

Another option is to add the interpreter here, (this is a temporary solution):

  "settings": {
    "yaml.schemas": {
      "https://gitlab.com/gitlab-org/gitlab/-/raw/master/app/assets/javascripts/editor/schema/ci.json": "file:///c%3A/Users/xxxxx/git/****-strats-ci/template-research-ci.yml"
    },
    "black-formatter.interpreter": ["c:/Users/xxx/AppData/Local/pypoetry/Cache/virtualenvs/monitoring-WTCPidQC-py3.10/Scripts/python"]
  },

This is scenario that is not currently handled well. I will need to investigate what we can do to provide a better fix for this in these cases.

That unfortunately doesn't work because I have different black versions requirements on different roots.

karthiknadig commented 1 year ago

That unfortunately doesn't work because I have different black versions requirements on different roots.

The root cause if this seems to be same as https://github.com/microsoft/vscode-black-formatter/issues/233 Closing this against that issue.

rdbisme commented 1 year ago

@karthiknadig are you sure about this? I'm pretty confident this configuration was working before. The problem is that right now it requires an interpreter set up before starting.

Also I have explicitly set up fromEnvironment, so the issue you mention would seem to not apply.

It stopped working very recently.

karthiknadig commented 1 year ago

I see. When you say recently was it in the last week? If it is last week, it could be due to python extension. You could revert python extension to 2023.6.1 and see if it helps.

karthiknadig commented 1 year ago

You can also try selecting a python version for this level: image

It need not be the python that you use for the workspace, we just need one to run the server.

karthiknadig commented 1 year ago

If none of that works, would it be possible for you to create a minimal repro with details on how and where you have environments. That will greatly help debug this. I tried creating a few different styles of multi-root repositories, I was not able to repro it.

FlorinAndrei commented 1 year ago

I have no environments, just the system-level Python, with Black installed with pip install --user black and today this extension has stopped working. It was working yesterday.

Windows 10 Python 3.11.3 Black 23.3.0

Python is installed for all users, it's in C:\Program Files\... and python.exe is in the PATH.

The notebooks work just fine with this Python interpreter.

No environments. No workspace. I just start VSCode, open a folder, then open a Jupyter Notebook in that folder. I cannot format the notebook.

The only recent change I'm aware of is the latest VSCode update to 1.78.2. I believe yesterday I was using the previous version.

2023-05-11 16:01:07.303 [info] Name: Black Formatter
2023-05-11 16:01:07.304 [info] Module: black
2023-05-11 16:01:07.304 [info] Python extension loading
2023-05-11 16:01:07.304 [info] Waiting for interpreter from python extension.
2023-05-11 16:01:09.417 [info] Python extension loaded
2023-05-11 16:01:09.427 [error] Python interpreter missing:
[Option 1] Select python interpreter using the ms-python.python.
[Option 2] Set an interpreter using "black-formatter.interpreter" setting.
Please use Python 3.7 or greater.
2023-05-11 16:06:06.168 [info] Formatting requested before server has started.
2023-05-11 16:07:50.329 [info] Formatting requested before server has started.
2023-05-11 16:09:11.814 [info] Formatting requested before server has started.
2023-05-11 16:09:11.815 [info] Formatting requested before server has started.
karthiknadig commented 1 year ago

@karrtikr We have not release a new version of the black extension in weeks, but seems like the ability to get python paths stopped working. Can you help investigate?

@FlorinAndrei what is the version of VS Code, and Python extension you have installed? did you by change update VS Code or python extension recently

FlorinAndrei commented 1 year ago

VSCode 1.78.2 Python 3.11.3 Black 23.3.0 Black Formatter v2023.2.0

Yeah, today I've applied the update to the latest VSCode. Now the extension is not working anymore.

It was working yesterday. I cannot think of any recent meaningful change other than the VSCode update.

karthiknadig commented 1 year ago

Can you try reverting the python extension to older version? image

And select 2023.6.1: image

See if that fixes this.

FlorinAndrei commented 1 year ago

Success! Black Formatter v2022.6.0 works just fine.

karthiknadig commented 1 year ago

@FlorinAndrei I actually want you to revert Python extension to 2023.6.1... keep Black at 2023.2.0

FlorinAndrei commented 1 year ago

Okay. Reverted Black Formatter to the latest version.

Downgraded the Python extension in VSCode to version v2023.6.1 - and now Black Formatter works just fine.

Re-updated the Python extension to the latest v2023.8.0 and Black Formatter still works just fine.

I now have all VSCode extensions updated to the latest and the formatter is working. I cannot reproduce the issue anymore.

rdbisme commented 1 year ago

Tested: Reverting Python extension to v2023.6.1 doesn't change the behaviour: 

2023-05-12 10:43:59.525 [info] Name: Black Formatter
2023-05-12 10:43:59.525 [info] Module: black
2023-05-12 10:43:59.525 [info] Python extension loading
2023-05-12 10:43:59.525 [info] Waiting for interpreter from python extension.
2023-05-12 10:44:15.666 [info] Python extension loaded
2023-05-12 10:44:15.708 [error] Python interpreter missing:
[Option 1] Select python interpreter using the ms-python.python.
[Option 2] Set an interpreter using "black-formatter.interpreter" setting.
Please use Python 3.7 or greater.

Reverting VScode to 1.77.3 also doesn't fix for me.

Reverting the Black formatter extension to v2022.6.0 fixes the problem for me. The server starts successfully and it's able to find black in the environments that have it (and fail for the others).

Name: Black Formatter
Module: black
Python extension loading
Waiting for interpreter from python extension.
Python extension loaded
CWD Server: c:\Users\xxx\git\qupy0
sys.path used to run Server:
   C:\Users\xxx\scoop\persist\vscode\data\extensions\ms-python.black-formatter-2022.6.0\bundled\tool
   C:\Users\xxx\scoop\apps\python310\current\python310.zip
   C:\Users\xxx\scoop\apps\python310\current\DLLs
   C:\Users\xxx\scoop\apps\python310\current\lib
   C:\Users\xxx\scoop\apps\python310\current
   c:\Users\xxx\AppData\Local\pypoetry\Cache\virtualenvs\qupy0
   c:\Users\xxx\AppData\Local\pypoetry\Cache\virtualenvs\qupy0\lib\site-packages
   C:\Users\xxx\git\qupy0\src
   c:\Users\xxx\AppData\Local\pypoetry\Cache\virtualenvs\qupy0\lib\site-packages\win32
   c:\Users\xxx\AppData\Local\pypoetry\Cache\virtualenvs\qupy0\lib\site-packages\win32\lib
   c:\Users\xxx\AppData\Local\pypoetry\Cache\virtualenvs\qupy0\lib\site-packages\Pythonwin
   C:\Users\xxx\scoop\persist\vscode\data\extensions\ms-python.black-formatter-2022.6.0\bundled\libs
Settings used to run Server:
[
    {
        "workspace": "file:///c%3A/Users/xxx/git/***/src/***_prod",
        "logLevel": "error",
        "args": [],
        "path": [],
        "interpreter": [],
        "importStrategy": "fromEnvironment",
        "showNotifications": "off"
    },
    {
        "workspace": "file:///c%3A/Users/xxx/git/***",
        "logLevel": "error",
        "args": [],
        "path": [],
        "interpreter": [
            "c:\\Users\\xxx\\AppData\\Local\\pypoetry\\Cache\\virtualenvs\\***-4-3s-KfG-py3.10\\Scripts\\python.exe"
        ],
        "importStrategy": "fromEnvironment",
        "showNotifications": "off"
    },
    {
        "workspace": "file:///c%3A/Users/xxx/git/***-deploy/src/qp",
        "logLevel": "error",
        "args": [],
        "path": [],
        "interpreter": [
            "c:\\Users\\xxx\\AppData\\Local\\pypoetry\\Cache\\virtualenvs\\qp-k9BKPGhN-py3.10\\Scripts\\python.exe"
        ],
        "importStrategy": "fromEnvironment",
        "showNotifications": "off"
    },
    {
        "workspace": "file:///c%3A/Users/xxx/git/***-strats-ci",
        "logLevel": "error",
        "args": [],
        "path": [],
        "interpreter": [
            "c:\\Users\\xxx\\AppData\\Local\\pypoetry\\Cache\\virtualenvs\\***-strats-ci-25nC5AIC-py3.10\\Scripts\\python.exe"
        ],
        "importStrategy": "fromEnvironment",
        "showNotifications": "off"
    },
    {
        "workspace": "file:///c%3A/Users/xxx/git/qupy0",
        "logLevel": "error",
        "args": [],
        "path": [],
        "interpreter": [
            "c:\\Users\\xxx\\AppData\\Local\\pypoetry\\Cache\\virtualenvs\\qupy0\\Scripts\\python.exe"
        ],
        "importStrategy": "fromEnvironment",
        "showNotifications": "off"
    },
    {
        "workspace": "file:///c%3A/Users/xxx/git/***-ci",
        "logLevel": "error",
        "args": [],
        "path": [],
        "interpreter": [
            "c:\\Users\\xxx\\scoop\\shims\\python3"
        ],
        "importStrategy": "fromEnvironment",
        "showNotifications": "off"
    },
    {
        "workspace": "file:///c%3A/Users/xxx/git/***-deploy/src/backfill-ci",
        "logLevel": "error",
        "args": [],
        "path": [],
        "interpreter": [
            "c:\\Users\\xxx\\AppData\\Local\\pypoetry\\Cache\\virtualenvs\\backfill-ci-6e_RSQmN-py3.10\\Scripts\\python.exe"
        ],
        "importStrategy": "fromEnvironment",
        "showNotifications": "off"
    },
    {
        "workspace": "file:///c%3A/Users/xxx/git/***-deploy/src/internal-docs",
        "logLevel": "error",
        "args": [],
        "path": [],
        "interpreter": [
            "c:\\Users\\xxx\\AppData\\Local\\pypoetry\\Cache\\virtualenvs\\***-internal-docs-ll0pM7nK-py3.10\\Scripts\\python.exe"
        ],
        "importStrategy": "fromEnvironment",
        "showNotifications": "off"
    },
    {
        "workspace": "file:///c%3A/Users/xxx/git/***-api",
        "logLevel": "error",
        "args": [],
        "path": [],
        "interpreter": [
            "c:\\Users\\xxx\\AppData\\Local\\pypoetry\\Cache\\virtualenvs\\app-kh5kM_7w-py3.10\\Scripts\\python.exe"
        ],
        "importStrategy": "fromEnvironment",
        "showNotifications": "off"
    },
    {
        "workspace": "file:///c%3A/Users/xxx/git/***-monitoring",
        "logLevel": "error",
        "args": [],
        "path": [],
        "interpreter": [
            "c:\\Users\\xxx\\AppData\\Local\\pypoetry\\Cache\\virtualenvs\\monitoring-WTCPidQC-py3.10\\Scripts\\python.exe"
        ],
        "importStrategy": "fromEnvironment",
        "showNotifications": "off"
    },
    {
        "workspace": "file:///c%3A/Users/xxx/git/***-deploy/src/cmd-batch",
        "logLevel": "error",
        "args": [],
        "path": [],
        "interpreter": [
            "c:\\Users\\xxx\\AppData\\Local\\pypoetry\\Cache\\virtualenvs\\cmd-batch-e6bUdhSH-py3.10\\Scripts\\python.exe"
        ],
        "importStrategy": "fromEnvironment",
        "showNotifications": "off"
    },
    {
        "workspace": "file:///c%3A/Users/xxx/git/***-ui",
        "logLevel": "error",
        "args": [],
        "path": [],
        "interpreter": [
            "c:\\Users\\xxx\\scoop\\shims\\python3"
        ],
        "importStrategy": "fromEnvironment",
        "showNotifications": "off"
    }
]

c:\Users\xxx\AppData\Local\pypoetry\Cache\virtualenvs\qupy0\Scripts\python.exe -m black --version
CWD formatter: c:\Users\xxx\git\***\src\***_prod
Version info for formatter running for c:\Users\xxx\git\***\src\***_prod:
black, 22.10.0 (compiled: no)
Python (CPython) 3.10.11

SUPPORTED black>=22.3.0
FOUND black==22.10.0

c:\Users\xxx\AppData\Local\pypoetry\Cache\virtualenvs\***-4-3s-KfG-py3.10\Scripts\python.exe -m black --version
CWD formatter: c:\Users\xxx\git\***
Version info for formatter running for c:\Users\xxx\git\***:
black, 22.12.0 (compiled: yes)
Python (CPython) 3.10.11

SUPPORTED black>=22.3.0
FOUND black==22.12.0

c:\Users\xxx\AppData\Local\pypoetry\Cache\virtualenvs\qp-k9BKPGhN-py3.10\Scripts\python.exe -m black --version
CWD formatter: c:\Users\xxx\git\***-deploy\src\qp
Version info for formatter running for c:\Users\xxx\git\***-deploy\src\qp:
black, 23.3.0 (compiled: yes)
Python (CPython) 3.10.11

SUPPORTED black>=22.3.0
FOUND black==23.3.0

c:\Users\xxx\AppData\Local\pypoetry\Cache\virtualenvs\***-strats-ci-25nC5AIC-py3.10\Scripts\python.exe -m black --version
CWD formatter: c:\Users\xxx\git\***-strats-ci
Version info for formatter running for c:\Users\xxx\git\***-strats-ci:
black, 23.3.0 (compiled: yes)
Python (CPython) 3.10.11

SUPPORTED black>=22.3.0
FOUND black==23.3.0

c:\Users\xxx\AppData\Local\pypoetry\Cache\virtualenvs\qupy0\Scripts\python.exe -m black --version
CWD formatter: c:\Users\xxx\git\qupy0
Version info for formatter running for c:\Users\xxx\git\qupy0:
black, 22.10.0 (compiled: no)
Python (CPython) 3.10.11

SUPPORTED black>=22.3.0
FOUND black==22.10.0

c:\Users\xxx\scoop\shims\python3 -m black --version
CWD formatter: c:\Users\xxx\git\***-ci
[Error - 10:47:28 AM] Traceback (most recent call last):
  File "C:\Users\xxx\scoop\persist\vscode\data\extensions\ms-python.black-formatter-2022.6.0\bundled\tool\runner.py", line 57, in <module>
    result = utils.run_module(
  File "C:\Users\xxx\scoop\persist\vscode\data\extensions\ms-python.black-formatter-2022.6.0\bundled\tool\utils.py", line 141, in run_module
    return _run_module(module, argv, use_stdin, source)
  File "C:\Users\xxx\scoop\persist\vscode\data\extensions\ms-python.black-formatter-2022.6.0\bundled\tool\utils.py", line 128, in _run_module
    runpy.run_module(module, run_name="__main__")
  File "C:\Users\xxx\scoop\apps\python310\current\lib\runpy.py", line 220, in run_module
    mod_name, mod_spec, code = _get_module_details(mod_name)
  File "C:\Users\xxx\scoop\apps\python310\current\lib\runpy.py", line 140, in _get_module_details
    raise error("No module named %s" % mod_name)
ImportError: No module named black

Version info for formatter running for c:\Users\xxx\git\***-ci:

Error while detecting black version:
Traceback (most recent call last):
  File "C:\Users\xxx\scoop\persist\vscode\data\extensions\ms-python.black-formatter-2022.6.0\bundled\tool\server.py", line 231, in _log_version_info
    first_line = result.stdout.splitlines(keepends=False)[0]
IndexError: list index out of range

c:\Users\xxx\AppData\Local\pypoetry\Cache\virtualenvs\backfill-ci-6e_RSQmN-py3.10\Scripts\python.exe -m black --version
CWD formatter: c:\Users\xxx\git\***-deploy\src\backfill-ci
Version info for formatter running for c:\Users\xxx\git\***-deploy\src\backfill-ci:
black, 22.10.0 (compiled: yes)
Python (CPython) 3.10.11

SUPPORTED black>=22.3.0
FOUND black==22.10.0

c:\Users\xxx\AppData\Local\pypoetry\Cache\virtualenvs\***-internal-docs-ll0pM7nK-py3.10\Scripts\python.exe -m black --version
CWD formatter: c:\Users\xxx\git\***-deploy\src\internal-docs
[Error - 10:47:28 AM] Traceback (most recent call last):
  File "C:\Users\xxx\scoop\persist\vscode\data\extensions\ms-python.black-formatter-2022.6.0\bundled\tool\runner.py", line 57, in <module>
    result = utils.run_module(
  File "C:\Users\xxx\scoop\persist\vscode\data\extensions\ms-python.black-formatter-2022.6.0\bundled\tool\utils.py", line 141, in run_module
    return _run_module(module, argv, use_stdin, source)
  File "C:\Users\xxx\scoop\persist\vscode\data\extensions\ms-python.black-formatter-2022.6.0\bundled\tool\utils.py", line 128, in _run_module
    runpy.run_module(module, run_name="__main__")
  File "C:\Users\xxx\scoop\apps\python310\current\lib\runpy.py", line 220, in run_module
    mod_name, mod_spec, code = _get_module_details(mod_name)
  File "C:\Users\xxx\scoop\apps\python310\current\lib\runpy.py", line 140, in _get_module_details
    raise error("No module named %s" % mod_name)
ImportError: No module named black

Version info for formatter running for c:\Users\xxx\git\***-deploy\src\internal-docs:

Error while detecting black version:
Traceback (most recent call last):
  File "C:\Users\xxx\scoop\persist\vscode\data\extensions\ms-python.black-formatter-2022.6.0\bundled\tool\server.py", line 231, in _log_version_info
    first_line = result.stdout.splitlines(keepends=False)[0]
IndexError: list index out of range

c:\Users\xxx\AppData\Local\pypoetry\Cache\virtualenvs\app-kh5kM_7w-py3.10\Scripts\python.exe -m black --version
CWD formatter: c:\Users\xxx\git\***-api
Version info for formatter running for c:\Users\xxx\git\***-api:
black, 23.3.0 (compiled: yes)
Python (CPython) 3.10.11

SUPPORTED black>=22.3.0
FOUND black==23.3.0

c:\Users\xxx\AppData\Local\pypoetry\Cache\virtualenvs\monitoring-WTCPidQC-py3.10\Scripts\python.exe -m black --version
CWD formatter: c:\Users\xxx\git\***-monitoring
Version info for formatter running for c:\Users\xxx\git\***-monitoring:
black, 23.3.0 (compiled: yes)
Python (CPython) 3.10.11

SUPPORTED black>=22.3.0
FOUND black==23.3.0

c:\Users\xxx\AppData\Local\pypoetry\Cache\virtualenvs\cmd-batch-e6bUdhSH-py3.10\Scripts\python.exe -m black --version
CWD formatter: c:\Users\xxx\git\***-deploy\src\cmd-batch
Version info for formatter running for c:\Users\xxx\git\***-deploy\src\cmd-batch:
black, 23.3.0 (compiled: yes)
Python (CPython) 3.10.11

SUPPORTED black>=22.3.0
FOUND black==23.3.0

c:\Users\xxx\scoop\shims\python3 -m black --version
CWD formatter: c:\Users\xxx\git\***-ui
[Error - 10:47:31 AM] Traceback (most recent call last):
  File "C:\Users\xxx\scoop\persist\vscode\data\extensions\ms-python.black-formatter-2022.6.0\bundled\tool\runner.py", line 57, in <module>
    result = utils.run_module(
  File "C:\Users\xxx\scoop\persist\vscode\data\extensions\ms-python.black-formatter-2022.6.0\bundled\tool\utils.py", line 141, in run_module
    return _run_module(module, argv, use_stdin, source)
  File "C:\Users\xxx\scoop\persist\vscode\data\extensions\ms-python.black-formatter-2022.6.0\bundled\tool\utils.py", line 128, in _run_module
    runpy.run_module(module, run_name="__main__")
  File "C:\Users\xxx\scoop\apps\python310\current\lib\runpy.py", line 220, in run_module
    mod_name, mod_spec, code = _get_module_details(mod_name)
  File "C:\Users\xxx\scoop\apps\python310\current\lib\runpy.py", line 140, in _get_module_details
    raise error("No module named %s" % mod_name)
ImportError: No module named black

Version info for formatter running for c:\Users\xxx\git\***-ui:

Error while detecting black version:
Traceback (most recent call last):
  File "C:\Users\xxx\scoop\persist\vscode\data\extensions\ms-python.black-formatter-2022.6.0\bundled\tool\server.py", line 231, in _log_version_info
    first_line = result.stdout.splitlines(keepends=False)[0]
IndexError: list index out of range

The problem seems black extension to me @karthiknadig

rdbisme commented 1 year ago

I confirm old black extension (v2022.6.0) and new Python extension works also Ok, both on Vscode 1.77.x and 1.78.x.

karthiknadig commented 1 year ago

@rdbisme Now that it started working with the old black, if you switch back to newer version does it work?

karrtikr commented 1 year ago

@rdbisme Looks like an invalid interpreter was selected was for the workspace file:///c%3A/Users/xxx/git/***/src/***_prod:

image

If you're able to reproduce the issue again with latest Black and latest Python extension, kindly provide the following:

Output for Python in the Output panel (View→Output, change the drop-down the upper-right of the Output panel to Python)

``` XXX ```

We very much appreciated!

FlorinAndrei commented 1 year ago

The problem is occurring again.

I need this extension to simply work. I don't have time for this. So I am going to uninstall it and revert to the old formatter until the issues are sorted out.

karthiknadig commented 1 year ago

Moving this to python extension. This is an issue with environment selection. The error is shown when Python extension does not provide a python path for the current workspace.

rdbisme commented 1 year ago

@karthiknadig I don't think your assessment is correct. If I revert to the previous extension version, things works as expected. The extension is able to get the intepreter for those projects in the multi root where it's correctly defined (and fail in the other ones).

There's clearly a bug here where when fromEnvironment is defined, the latest extension version is asking for an intepreter path even if it should get it from the virtual environment instead.

Again, on latest python extension version, reverting black extension to the previous one makes thing work again (and it's able to correctly introspect the interpreter).

I believe it's a bug of this extension, not Python one.

rdbisme commented 1 year ago

@rdbisme Looks like an invalid interpreter was selected was for the workspace file:///c%3A/Users/xxx/git/***/src/***_prod:

image

This is normal. The intepreter is empty for the *_prod project because it actually doesn't have any selected. It's not configured for that project. Also, the problem I'm having is with the -monitoring project, not _prod one. And that one clearly has an intepreter defined . 

{
        "workspace": "file:///c%3A/Users/xxx/git/***-monitoring",
        "logLevel": "error",
        "args": [],
        "path": [],
        "interpreter": [
            "c:\\Users\\xxx\\AppData\\Local\\pypoetry\\Cache\\virtualenvs\\monitoring-WTCPidQC-py3.10\\Scripts\\python.exe"
        ],
        "importStrategy": "fromEnvironment",
        "showNotifications": "off"
    },

Also, again, reverting the extension back to the previous version makes everything work as expected.

karthiknadig commented 1 year ago

@rdbisme The following error is only thrown when, we ask for interpreter from python extension to start the server script and we don't get one. See below for my hypothesis on this issue and why I moved it here for further investigation.

2023-05-10 18:13:39.408 [error] Python interpreter missing:
[Option 1] Select python interpreter using the ms-python.python.
[Option 2] Set an interpreter using "black-formatter.interpreter" setting.
Please use Python 3.7 or greater.

Here is the difference between the two versions:

Black extension 2022.6

In version 2022.6, we got the interpreter to start the server by asking python extension for interpreter for the first workspace folder by specifically passing it as a scope argument, equivalent to:

const interpreterDetails = await getInterpreterDetails( getProjectRoot());

ref: Server start code: https://github.com/microsoft/vscode-black-formatter/blob/7b90eff26f0ee0bccbf87dfe541bfeaf51d2140a/src/common/server.ts#L86 Getting interpreter per workspace https://github.com/microsoft/vscode-black-formatter/blob/7b90eff26f0ee0bccbf87dfe541bfeaf51d2140a/src/common/settings.ts#L79

Black extension 2023.2

In version 2023.2 we got the interpreter to start the server by asking python extension by using undefined as scope. The undefined here was meant to represent the entire workspace.

const interpreterDetails = await getInterpreterDetails();

ref: https://github.com/microsoft/vscode-black-formatter/blob/e3e5ad1bbb9724a0a0a528c167437915d669ea87/src/extension.ts#L54

Hypothesis

I moved it here because there were changes to the API to get python, and wanted to make sure there wasn't a bug in this approach to querying for interpreters. If this is a bug in the python extension API, then it will be fixed here. If this is a bug in the black extension, as in it should always be passing scope, then I will move it back and revert to always using scope.

Additionally, black extension stable release was early April. With this issue being reported recently my hypothesis was that when python extension pushed a new release last week this might have started to occur. It works with the older version, because the code path in python extension that handles scoped interpreters still works as expected, but there might be a bug when we ask for interpreters without scope.

This is the API reference: https://github.com/microsoft/vscode-python/blob/a74f1d15b58e2a835f5564449918452ff30d62d6/src/client/apiTypes.ts#L61-L67

getInterpreterDetails calls into python extension API getActiveEnvironmentPath which I think is not returning the expected interpreter when undefined is used.

This is the implementation of getInterpreterDetails:

export async function getInterpreterDetails(resource?: Uri): Promise<IInterpreterDetails> {
    const api = await getPythonExtensionAPI();
    const environment = await api?.environments.resolveEnvironment(
        api?.environments.getActiveEnvironmentPath(resource),
    );
    if (environment?.executable.uri && checkVersion(environment)) {
        return { path: [environment?.executable.uri.fsPath], resource };
    }
    return { path: undefined, resource };
}

Reasoning behind the change from using scoped to un-scoped

Python extension allows users to select interpreter at the workspace level and at workspace folder level. By always scoping to the first workspace in the black extension meant that it did not honor the semantics of selection that was used in the python extension. If it turns out the old way was better I will move it back there.

Details on fromEnvironment

Lastly, "black-formatter.importStrategy": "fromEnvironment" does not mean that black formatter will automatically select the virtual environment. It means that it will use black that is in the environment that the server is running in. black extension relies on python extension for interpreter paths in both useBundled vs fromEnvironment case.

For single-workspace: https://github.com/microsoft/vscode-black-formatter/blob/release/2023.2/bundled/tool/lsp_server.py#L19-L25

For multi-root: https://github.com/microsoft/vscode-black-formatter/blob/release/2023.2/bundled/tool/lsp_runner.py#L19-L23

It literally only defines where we insert the path to bundled black, like this. (path_to_add is path to bundled black):

        if strategy == "useBundled":
            sys.path.insert(0, path_to_add)
        elif strategy == "fromEnvironment":
            sys.path.append(path_to_add)
github-actions[bot] commented 1 year ago

Because we have not heard back with the information we requested, we are closing this issue for now. If you are able to provide the info later on, then we will be happy to re-open this issue to pick up where we left off.

Happy Coding!

github-actions[bot] commented 1 year ago

Because we have not heard back with the information we requested, we are closing this issue for now. If you are able to provide the info later on, then we will be happy to re-open this issue to pick up where we left off.

Happy Coding!

karthiknadig commented 1 year ago

This is mitigated in the black-formatter extension. Keeping this open to get a solution in general for these scenarios.

karrtikr commented 1 year ago

Discussing offline, but I think the right solution is to use https://github.com/microsoft/vscode-python/wiki/Python-Environment-APIs#get-current-known-environments to get all environments in the workspace and choose any one to launch LSP.

What is the expectation from the API when passing resource as undefined? As it is synchronous, unfortunately I think the only thing we can do is return the interpreter path set by the first workspace.

karrtikr commented 1 year ago

Note passing resource is not needed when using environments.known, which is preciously what is needed I think.

karrtikr commented 1 year ago

Summary:

For multi root scenarios, when no resource is passed into getActiveEnvironment() API, return the environment selected for the workspace, which will be:

"Auto-selected workspace environment" is defined as the preferred environment in the union of environments discovered in the following scopes: