Open aatle opened 4 months ago
Hi @aatle Thanks for filing this issue.
Can you opt out from the new terminal environment variable experiment for now and see if the problem persist?
You can do this by adding "python.experiments.optOutFrom": ["pythonTerminalEnvVarActivation"],
into your settings.json
As I have said in the original post, that setting is indeed a workaround and fixes the problem. Unfortunately, it is actually not a perfect solution because it also introduces a different problem.
(Unrelated to the setting), the Python extension will sometimes still activate the poetry virtual environment but decide to only edit (and corrupt) the PATH
variable, instead of editing a bunch of variables. Not sure why.
For the latter Project 2, the setting indeed fixes things, no environment variables are corrupted, and the extension does not contribute to terminal environment.\
\
For the former, the setting seems to corrupt PATH
even more, so sed
and other commands are not found, rendering the terminal almost useless. The environment changes show that the Python extension is still contributing PATH
, but has a different value.
Hey @anthonykim1, this issue might need further attention.
@aatle, you can help us out by closing this issue if the problem no longer exists, or adding more information.
This issue still exists, and makes all git
commands fail on some of my poetry projects.
but decide to only edit (and corrupt) the PATH variable, instead of editing a bunch of variables. Not sure why.
I believe this is part of the Terminal Env Var experiment as we prepend PATH and calculate other environment variables from user. https://github.com/microsoft/vscode-python/pull/22905
When you say "corrupted" PATH, are you looking the prepending pattern?
Yes, in PATH
, I am mainly looking at the prepended paths that have backslashes in them.
My original post isn't entirely clear, I'll clarify and summarize what I think is happening:
I am using Windows, poetry virtual environment, and Git Bash terminal. From my understanding, the opt-out experimental feature edits and calculates certain environment variables.
The feature has one problem: in the environment variables edited, some backslash literals (often present in Windows paths) become special escape sequences e.g. \v
, \n
. Only certain characters such as a
and n
can form escape sequences with backslash, so not all backslashes are corrupted.
So, paths in the edited environment variables such as C:\Program Files\Git\bin
become C:\Program Files\Giin
where there is a \b
backspace character in it. This obviously renders those paths useless/corrupted.
This problem is worse if the user's $HOME
variable is also affected. C:\Users\abcde
(calculated) -> C:\Usersbcde
with special \a
character. (Without Python extension env var experiment, it would be /c/Users/abcde
, using forward slashes.)
So, I think that there is a backslash issue somewhere in the process of parsing, editing, and exporting environment variable strings.
If I opt out of this feature using the setting, then some of my other environments, which look slightly different for some reason, get even more corrupted such that basic commands like sed
are not found and PATH
is corrupted.
@aatle Apologies for late response and thanks for further explanation here.
paths in the edited environment variables such as C:\Program Files\Git\bin become C:\Program Files\Giin where there is a \b backspace character in it.
This is interesting. Are you able to click on show environment contribution and see if WHO (for example Python extension specifically) is contributing to your corrupted environment?
Also note that GitBash is currently not officially supported by Python extension at this moment: https://github.com/microsoft/vscode-python/issues/23382
If I opt out of this feature using the setting, then some of my other environments, which look slightly different for some reason, get even more corrupted such that basic commands like sed are not found and PATH is corrupted.
If this is the case, then the problem may reside outside of Python extension. What the Terminal Env Variable experiment does is, it tries to "activate" your environment using environment variables instead of explicitly running activate script on your terminal. You will see the "activate" line of command in terminal if you opt out of this experiment, but if that STILL causes a problem then it becomes very hard to tell because Python extension would not mingle around with your PATH if you opt-out of the experiment.
Are you able to "correctly" activate your environment outside of VS Code environment? (External terminal for example)
@anthonykim1 No worries, my workflow has not been inconvenienced by this. (Also, I did not know that GitBash is not officially supported currently, I use it as default on Windows. Hopefully it is officially supported soon.)
I am mostly certain that this bug is caused by the Microsoft Python extension, in certain conditions.
The other shells I tested, powershell, cmd, GitBash outside of VS Code, they seemed to work fine when using poetry shell
venv.
Checking the Environment Contributors -> Show Environment Contributions, I see this file:
The invalid environment variables are: HOME
, EXEPATH
, and a part of PATH
.
The only other environment contributor is the git extension, and their variables seem fine. The issue does not occur when the Python extension is turned off.
I realized from more testing, it is indeed fixable without more issues by opting out of the terminal var experiment with settings.json. Turning off the feature makes the Python extension not contribute to the environment using variables, and instead uses activation script as you said.
"python.experiments.optOutFrom": ["pythonTerminalEnvVarActivation"],
Previously I said that this setting caused even worse errors on some projects, but it seems that this only applied to one specific older project I have, which I fixed. The issue will also only manifest if the selected python interpreter ("Python: Select Interpreter" command) is the poetry virtual environment one, not the global python interpreter, otherwise the venv doesn't get activated.
Conditions for bug, updated:
--> Causes certain Python extension environment variable contributions to be invalid because of backslashes in paths
Hey @anthonykim1, this issue might need further attention.
@aatle, you can help us out by closing this issue if the problem no longer exists, or adding more information.
Type: Bug
Behaviour
In the terminal under specific circumstances, some environment variables are loaded with invalid values because of backslashes, causing certain commands to fail.
It is expected that the environment variable values would not have special escape sequences from backslashes in Windows paths.
Steps to reproduce:
VS Code Python extension loading poetry virtual environment on Git Bash on Windows
C:\Users
, should start with lowercasea
, e.g. "abcde". (Some other characters probably also work for this: b,e,f,n,r,t,v.)poetry
installed (1.8.3 used, but probably any version) usingpip
.pyproject.toml
file with the following contents:[tool.poetry.dependencies] python = "^3.10"
[build-system] requires = ["poetry-core"] build-backend = "poetry.core.masonry.api"
2024-06-12 18:06:58.366 [info] Experiment 'pythonaacf' is active 2024-06-12 18:06:58.366 [info] Experiment 'pythonCreateEnvOnPipInstall' is active 2024-06-12 18:06:58.366 [info] Experiment 'pythonRecommendTensorboardExt' is active 2024-06-12 18:06:58.366 [info] Experiment 'pythonTerminalEnvVarActivation' is active 2024-06-12 18:06:58.366 [info] Experiment 'pythonTestAdapter' is active 2024-06-12 18:06:58.366 [info] Default formatter is set to charliermarsh.ruff for workspace c:\Users\abcde\development\reproduce 2024-06-12 18:06:58.366 [info] Starting Environment refresh 2024-06-12 18:06:58.366 [info] Starting Environment refresh 2024-06-12 18:06:58.366 [info] > hatch env show --json 2024-06-12 18:06:58.366 [info] cwd: . 2024-06-12 18:06:58.366 [info] > poetry env list --full-path 2024-06-12 18:06:58.366 [info] cwd: . 2024-06-12 18:06:59.770 [info] Environments refresh finished (event): 1467 milliseconds 2024-06-12 18:06:59.770 [info] Environments refresh paths discovered: 1467 milliseconds 2024-06-12 18:06:59.771 [info] Environments refresh finished (event): 1467 milliseconds 2024-06-12 18:06:59.771 [info] Environments refresh paths discovered: 1467 milliseconds 2024-06-12 18:06:59.772 [info] Environment refresh took 1468 milliseconds 2024-06-12 18:06:59.772 [info] Environment refresh took 1469 milliseconds 2024-06-12 18:06:59.780 [info] > pyenv which python 2024-06-12 18:06:59.780 [info] cwd: . 2024-06-12 18:06:59.826 [info] Running installed packages checker: { id: 'C:\USERS\ABCDE\APPDATA\LOCAL\PYPOETRY\CACHE\VIRTUALENVS\PROJECT-3JZSXIOL-PY3.12\SCRIPTS\PYTHON.EXE', sysPrefix: 'C:\Users\abcde\AppData\Local\pypoetry\Cache\virtualenvs\project-3JzsXiol-py3.12', envType: 'Poetry', envName: 'project-3JzsXiol-py3.12', envPath: 'C:\Users\abcde\AppData\Local\pypoetry\Cache\virtualenvs\project-3JzsXiol-py3.12', path: 'C:\Users\abcde\AppData\Local\pypoetry\Cache\virtualenvs\project-3JzsXiol-py3.12\Scripts\python.exe', architecture: 1, sysVersion: undefined, version: { raw: '3.12.4', major: 3, minor: 12, patch: 4, build: [], prerelease: [ 'final', '0' ] }, displayName: "Python 3.12.4 ('project-3JzsXiol-py3.12')", detailedDisplayName: "Python 3.12.4 ('project-3JzsXiol-py3.12': Poetry)", type: 'Virtual' } c:\Users\abcde.vscode\extensions\ms-python.python-2024.8.0\python_files\installed_check.py c:\Users\abcde\development\reproduce\pyproject.toml 2024-06-12 18:06:59.826 [info] > ~\AppData\Local\pypoetry\Cache\virtualenvs\project-3JzsXiol-py3.12\Scripts\python.exe ~.vscode\extensions\ms-python.python-2024.8.0\python_files\installed_check.py .\pyproject.toml 2024-06-12 18:06:59.828 [info] Python interpreter path: ~\AppData\Local\pypoetry\Cache\virtualenvs\project-3JzsXiol-py3.12\Scripts\python.exe 2024-06-12 18:06:59.856 [info] > . ~/AppData/Local/pypoetry/Cache/virtualenvs/project-3JzsXiol-py3.12/Scripts/activate && echo 'e8b39361-0157-4923-80e1-22d70d46dee6' && python ~/.vscode/extensions/ms-python.python-2024.8.0/python_files/printEnvVariables.py 2024-06-12 18:06:59.856 [info] shell: bash 2024-06-12 18:07:00.080 [info] > ~\AppData\Local\Programs\Python\Python312\python.exe ~/.vscode/extensions/ms-python.python-2024.8.0/python_files/printEnvVariables.py 2024-06-12 18:07:00.080 [info] shell: bash 2024-06-12 18:07:00.137 [info] Setting environment variable EXEPATH in collection to C:\Program Files\Git\bin {"applyAtShellIntegration":true,"applyAtProcessCreation":true} 2024-06-12 18:07:00.137 [info] Setting environment variable HOME in collection to C:\Users\abcde {"applyAtShellIntegration":true,"applyAtProcessCreation":true} 2024-06-12 18:07:00.137 [info] Setting environment variable MSYS in collection to disable_pcon {"applyAtShellIntegration":true,"applyAtProcessCreation":true} 2024-06-12 18:07:00.137 [info] Setting environment variable MSYSTEM in collection to MINGW64 {"applyAtShellIntegration":true,"applyAtProcessCreation":true} 2024-06-12 18:07:00.137 [info] Prepending environment variable PATH in collection to c:\Users\abcde.vscode\extensions\ms-python.python-2024.8.0\python_files\deactivate\bash;C:\Users\abcde\AppData\Local\pypoetry\Cache\virtualenvs\project-3JzsXiol-py3.12\Scripts;C:\Program Files\Git\mingw64\bin;C:\Program Files\Git\usr\bin;C:\Users\abcde\bin;C:\WINDOWS\system32;C:\WINDOWS;C:\WINDOWS\System32\Wbem;C:\WINDOWS\System32\WindowsPowerShell\v1.0;C:\WINDOWS\System32\OpenSSH;C:\Program Files (x86)\Windows Kits\10\Windows Performance Toolkit;C:\Program Files\Git\cmd;C:\Users\abcde\AppData\Local\Programs\Python\Python312\Scripts;C:\Users\abcde\AppData\Local\Programs\Python\Python312;C:\Users\abcde\AppData\Local\Microsoft\WindowsApps;C:\Users\abcde\AppData\Local\Programs\Microsoft VS Code\bin;C:\users\abcde.local\bin; {"applyAtShellIntegration":true,"applyAtProcessCreation":true} 2024-06-12 18:07:00.138 [info] Setting environment variable PLINK_PROTOCOL in collection to ssh {"applyAtShellIntegration":true,"applyAtProcessCreation":true} 2024-06-12 18:07:00.138 [info] Prepending environment variable PS1 in collection with (project-py3.12) {"applyAtShellIntegration":true,"applyAtProcessCreation":false} 2024-06-12 18:07:00.138 [info] Setting environment variable PWD in collection to C:/Users/abcde/AppData/Local/Programs/Microsoft VS Code {"applyAtShellIntegration":true,"applyAtProcessCreation":true} 2024-06-12 18:07:00.138 [info] Setting environment variable PYTHONIOENCODING in collection to utf-8 {"applyAtShellIntegration":true,"applyAtProcessCreation":true} 2024-06-12 18:07:00.138 [info] Setting environment variable PYTHONUNBUFFERED in collection to 1 {"applyAtShellIntegration":true,"applyAtProcessCreation":true} 2024-06-12 18:07:00.138 [info] Setting environment variable TERM in collection to xterm-256color {"applyAtShellIntegration":true,"applyAtProcessCreation":true} 2024-06-12 18:07:00.138 [info] Setting environment variable VIRTUAL_ENV in collection to C:/Users/abcde/AppData/Local/pypoetry/Cache/virtualenvs/project-3JzsXiol-py3.12 {"applyAtShellIntegration":true,"applyAtProcessCreation":true} 2024-06-12 18:07:00.138 [info] Setting environment variable VIRTUAL_ENV_PROMPT in collection to project-py3.12 {"applyAtShellIntegration":true,"applyAtProcessCreation":true} 2024-06-12 18:07:00.138 [info] Send text to terminal: C:/Users/abcde/AppData/Local/Programs/Python/Python312/python.exe c:\Users\abcde.vscode\extensions\ms-python.python-2024.8.0\python_files\printEnvVariablesToFile.py c:\Users\abcde.vscode\extensions\ms-python.python-2024.8.0\python_files\deactivate\bash\envVars.txt 2024-06-12 18:07:00.797 [info] Starting Jedi language server for reproduce. 2024-06-12 18:07:30.143 [error] Failed to initialize deactivate script C:\Program Files\Git\bin\bash.exe [Error: "c:\Users\abcde.vscode\extensions\ms-python.python-2024.8.0\python_files\deactivate\bash\envVars.txt" file not created at Timeout. (c:\Users\abcde.vscode\extensions\ms-python.python-2024.8.0\out\client\extension.js:2:273846)
at listOnTimeout (node:internal/timers:573:17)
at process.processTimers (node:internal/timers:514:7)]
Three corrupted environment variables
Bash ```bash $ echo $HOME C:\Usersbcde $ echo $PATH c:\Usersbcde\.vscodetensions\ms-python.python-2024.8.0\python_files\deactivatash;C:\Usersbcde\AppData\Local\pypoetry\Cache irtualenvs\project-3JzsXiol-py3.12\Scripts;C:\Program Files\Git\mingw6in;C:\Program Files\Git\usin;C:\Usersnthin;C:\WINDOWS\system32;C:\WINDOWS;C:\WINDOWS\System32\Wbem;C:\WINDOWS\System32\WindowsPowerShell 1.0;C:\WINDOWS\System32\OpenSSH;C:\Program Files (x86)\Windows Kits\10\Windows Performance Toolkit;C:\Program Files\Git/c/Users/abcde/bin:/mingw64/bin:/usr/local/bin:/usr/bin:/bin:/mingw64/bin:/usr/bin:/c/Users/abcde/bin:/c/Users/abcde/.vscode/extensions/ms-python.python-2024.8.0/python_files/deactivate/bash:/c/Users/abcde/AppData/Local/pypoetry/Cache/virtualenvs/project-3JzsXiol-py3.12/Scripts:/mingw64/bin:/usr/bin:/c/Users/abcde/bin:/c/WINDOWS/system32:/c/WINDOWS:/c/WINDOWS/System32/Wbem:/c/WINDOWS/System32/WindowsPowerShell/v1.0:/c/WINDOWS/System32/OpenSSH:/c/Program Files (x86)/Windows Kits/10/Windows Performance Toolkit:/cmd:/c/Users/abcde/AppData/Local/Programs/Python/Python312/Scripts:/c/Users/abcde/AppData/Local/Programs/Python/Python312:/c/Users/abcde/AppData/Local/Microsoft/WindowsApps:/c/Users/abcde/AppData/Local/Programs/Microsoft VS Code/bin:/c/users/abcde/.local/bin:/c/WINDOWS/system32:/c/WINDOWS:/c/WINDOWS/System32/Wbem:/c/WINDOWS/System32/WindowsPowerShell/v1.0:/c/WINDOWS/System32/OpenSSH:/c/Program Files (x86)/Windows Kits/10/Windows Performance Toolkit:/cmd:/c/Users/abcde/AppData/Local/Programs/Python/Python312/Scripts:/c/Users/abcde/AppData/Local/Programs/Python/Python312:/c/Users/abcde/AppData/Local/Microsoft/WindowsApps:/c/Users/abcde/AppData/Local/Programs/Microsoft VS Code/bin:/c/users/abcde/.local/bin:/usr/bin/vendor_perl:/usr/bin/core_perl $ echo $EXEPATH C:\Program Files\Giin ``` Python (shows escape sequences) ```bash $ python Python 3.12.4 (tags/v3.12.4:8e8a4ba, Jun 6 2024, 19:30:16) [MSC v.1940 64 bit (AMD64)] on win32 Type "help", "copyright", "credits" or "license" for more information. >>> import os >>> os.environ["HOME"] 'C:\\Users\x07bcde' >>> os.environ["EXEPATH"] 'C:\\Program Files\\Git\x08in' >>> os.environ["PATH"] 'c;C:\\Users\x07bcde\\.vscode\x1bxtensions\\ms-python.python-2024.8.0\\python_files\\deactivate\x08ash;C;C:\\Users\x07bcde\\AppData\\Local\\pypoetry\\Cache\x0birtualenvs\\project-3JzsXiol-py3.12\\Scripts;C;C:\\Program Files\\Git\\mingw64\x08in;C;C:\\Program Files\\Git\\usr\x08in;C;C:\\Users\x07bcde\x08in;C;C:\\WINDOWS\\system32;C;C:\\WINDOWS;C;C:\\WINDOWS\\System32\\Wbem;C;C:\\WINDOWS\\System32\\WindowsPowerShell\x0b1.0;C;C:\\WINDOWS\\System32\\OpenSSH;C;C:\\Program Files (x86)\\Windows Kits\\10\\Windows Performance Toolkit;C;C:\\Program Files\\Git\\c\\Users\\abcde\\bin;C:\\Program Files\\Git\\mingw64\\bin;C:\\Program Files\\Git\\usr\\local\\bin;C:\\Program Files\\Git\\usr\\bin;C:\\Program Files\\Git\\usr\\bin;C:\\Program Files\\Git\\mingw64\\bin;C:\\Program Files\\Git\\usr\\bin;C:\\Users\\abcde\\bin;C:\\Users\\abcde\\.vscode\\extensions\\ms-python.python-2024.8.0\\python_files\\deactivate\\bash;C:\\Users\\abcde\\AppData\\Local\\pypoetry\\Cache\\virtualenvs\\project-3JzsXiol-py3.12\\Scripts;C:\\Program Files\\Git\\mingw64\\bin;C:\\Program Files\\Git\\usr\\bin;C:\\Users\\abcde\\bin;C:\\WINDOWS\\system32;C:\\WINDOWS;C:\\WINDOWS\\System32\\Wbem;C:\\WINDOWS\\System32\\WindowsPowerShell\\v1.0;C:\\WINDOWS\\System32\\OpenSSH;C:\\Program Files (x86)\\Windows Kits\\10\\Windows Performance Toolkit;C:\\Program Files\\Git\\cmd;C:\\Users\\abcde\\AppData\\Local\\Programs\\Python\\Python312\\Scripts;C:\\Users\\abcde\\AppData\\Local\\Programs\\Python\\Python312;C:\\Users\\abcde\\AppData\\Local\\Microsoft\\WindowsApps;C:\\Users\\abcde\\AppData\\Local\\Programs\\Microsoft VS Code\\bin;C:\\users\\abcde\\.local\\bin;C:\\WINDOWS\\system32;C:\\WINDOWS;C:\\WINDOWS\\System32\\Wbem;C:\\WINDOWS\\System32\\WindowsPowerShell\\v1.0;C:\\WINDOWS\\System32\\OpenSSH;C:\\Program Files (x86)\\Windows Kits\\10\\Windows Performance Toolkit;C:\\Program Files\\Git\\cmd;C:\\Users\\abcde\\AppData\\Local\\Programs\\Python\\Python312\\Scripts;C:\\Users\\abcde\\AppData\\Local\\Programs\\Python\\Python312;C:\\Users\\abcde\\AppData\\Local\\Microsoft\\WindowsApps;C:\\Users\\abcde\\AppData\\Local\\Programs\\Microsoft VS Code\\bin;C:\\users\\abcde\\.local\\bin;C:\\Program Files\\Git\\usr\\bin\\vendor_perl;C:\\Program Files\\Git\\usr\\bin\\core_perl' ```
Notes
It would seem that the Python extension is attempting to load environment variables with values that include backslashes because of Windows paths. But somewhere in the process, a few of the backslashes are being interpreted as escape sequences. For example
\v
becomes a vertical tab and\b
becomes a backspace. The interpretation aligns with how a.env
file interprets backslashes. See C Escape Sequences.\ If the user does not have a user account name that starts with a valid escape sequence character, only PATH is corrupted, and not enough to be immediately noticeable.\A workaround is adding
"python.experiments.optOutFrom": ["pythonTerminalEnvVarActivation"]
to settings.json.Extension version: 2024.8.0 VS Code version: Code 1.90.0 (89de5a8d4d6205e5b11647eb6a74844ca23d2573, 2024-06-04T19:33:54.889Z) OS version: Windows_NT x64 10.0.22631 Modes:
python.languageServer
setting: DefaultUser Settings
``` languageServer: "Jedi" experiments • optOutFrom: ["pythonTerminalEnvVarActivation"] ```
Installed Extensions
|Extension Name|Extension Id|Version| |---|---|---| |Azure Repos|ms-vscode.azure-repos|0.38.0| |Even Better TOML|tamasfe.even-better-toml|0.19.2| |GitHub Pull Requests|GitHub.vscode-pull-request-github|0.88.1| |GitHub Repositories|GitHub.remotehub|0.62.0| |JavaScript Debugger|ms-vscode.js-debug|1.90.0| |JavaScript Debugger Companion Extension|ms-vscode.js-debug-companion|1.1.2| |Mypy|matangover.mypy|0.3.1| |Python|ms-python.python|2024.8.0| |Python Debugger|ms-python.debugpy|2024.6.0| |Remote Repositories|ms-vscode.remote-repositories|0.40.0| |Ruff|charliermarsh.ruff|2024.26.0| |Table Visualizer for JavaScript Profiles|ms-vscode.vscode-js-profile-table|1.0.9|System Info
|Item|Value| |---|---| |CPUs|11th Gen Intel(R) Core(TM) i7-11700F @ 2.50GHz (16 x 2496)| |GPU Status|2d_canvas: enabledcanvas_oop_rasterization: enabled_on
direct_rendering_display_compositor: disabled_off_ok
gpu_compositing: enabled
multiple_raster_threads: enabled_on
opengl: enabled_on
rasterization: enabled
raw_draw: disabled_off_ok
skia_graphite: disabled_off
video_decode: enabled
video_encode: enabled
vulkan: disabled_off
webgl: enabled
webgl2: enabled
webgpu: enabled| |Load (avg)|undefined| |Memory (System)|15.77GB (6.93GB free)| |Process Argv|--crash-reporter-id 937dcfed-06f3-46d6-bf2d-b1f57e7b00a9| |Screen Reader|no| |VM|0%|
A/B Experiments
``` vsliv368cf:30146710 vspor879:30202332 vspor708:30202333 vspor363:30204092 vscorecescf:30445987 vscod805cf:30301675 binariesv615:30325510 vsaa593:30376534 py29gd2263:31024239 c4g48928:30535728 azure-dev_surveyone:30548225 962ge761:30959799 pythongtdpath:30769146 welcomedialog:30910333 pythonidxpt:30866567 pythonnoceb:30805159 asynctok:30898717 pythontestfixt:30902429 pythonregdiag2:30936856 pythonmypyd1:30879173 h48ei257:31000450 pythontbext0:30879054 accentitlementsc:30995553 dsvsc016:30899300 dsvsc017:30899301 dsvsc018:30899302 cppperfnew:31000557 dsvsc020:30976470 pythonait:31006305 jchc7451:31067544 chatpanelc:31048052 dsvsc021:30996838 f3je6385:31013174 pythoncenvpt:31062603 a69g1124:31058053 dvdeprecation:31068756 pythonprt:31056678 dwnewjupytercf:31046870 newcmakeconfigv2:31071590 26j00206:31048877 ```