search

microsoft / vscode-python

Python extension for Visual Studio Code
https://aka.ms/pvsc-marketplace
MIT License
4.25k stars 1.15k forks source link

Replaces User home with ~ #23533

Closed broganrossae closed 6 days ago

broganrossae commented 1 month ago

Type: Bug

Behaviour

When using "python.testing.pytestArgs" any paths to the user's home directory get converted to ~. Using ${env:HOME} results in the same thing.

Steps to reproduce:

In my case I was using pytest's '-o' option to extend module search directories. So changing the workspace settings to:

{
  "settings": {
    "python.testing.pytestArgs: [
      "-o 'pythonpath=/User/user.name/repos/extra/path'"
    ]
  }
}

Then when running test discovery (Cmd Palette: Python: Refresh Tests), and investigating the output I see that something has converted /Users/user.name to ~. Which, apparently, causes my test discovery to fail.

Diagnostic data

Output for Python in the Output panel (View→Output, change the drop-down the upper-right of the Output panel to Python)

``` 2024-05-31 08:44:45.142 [info] > . ~/repos/repoName/.venv/bin/activate && echo 'e8b39361-0157-4923-80e1-22d70d46dee6' && python ~/.vscode/extensions/ms-python.python-2024.6.0/python_files/printEnvVariables.py 2024-05-31 08:44:45.143 [info] shell: zsh 2024-05-31 08:44:45.144 [info] > . ~/repos/repoName/.venv/bin/activate && echo 'e8b39361-0157-4923-80e1-22d70d46dee6' && python ~/.vscode/extensions/ms-python.python-2024.6.0/python_files/printEnvVariables.py 2024-05-31 08:44:45.144 [info] shell: zsh 2024-05-31 08:44:45.237 [info] Prepending environment variable PATH in collection with /Users/user.name/.vscode/extensions/ms-python.python-2024.6.0/python_files/deactivate/zsh:/Users/user.name/repos/repoName/.venv/bin: {"applyAtShellIntegration":true,"applyAtProcessCreation":true} 2024-05-31 08:44:45.238 [info] Setting environment variable VIRTUAL_ENV in collection to /Users/user.name/repos/repoName/.venv {"applyAtShellIntegration":true,"applyAtProcessCreation":true} 2024-05-31 08:44:45.238 [info] Setting environment variable VIRTUAL_ENV_PROMPT in collection to .venv {"applyAtShellIntegration":true,"applyAtProcessCreation":true} 2024-05-31 08:44:45.238 [info] Prepending environment variable PS1 in collection with .venv {"applyAtShellIntegration":true,"applyAtProcessCreation":false} 2024-05-31 08:44:45.239 [info] Prepending environment variable PATH in collection with /Users/user.name/.vscode/extensions/ms-python.python-2024.6.0/python_files/deactivate/zsh:/Users/user.name/repos/repoName/.venv/bin: {"applyAtShellIntegration":true,"applyAtProcessCreation":true} 2024-05-31 08:44:45.239 [info] Setting environment variable VIRTUAL_ENV in collection to /Users/user.name/repos/repoName/.venv {"applyAtShellIntegration":true,"applyAtProcessCreation":true} 2024-05-31 08:44:45.240 [info] Setting environment variable VIRTUAL_ENV_PROMPT in collection to .venv {"applyAtShellIntegration":true,"applyAtProcessCreation":true} 2024-05-31 08:44:45.240 [info] Prepending environment variable PS1 in collection with .venv {"applyAtShellIntegration":true,"applyAtProcessCreation":false} 2024-05-31 08:44:48.459 [info] Discover tests for workspace name: repoName - uri: /Users/user.name/repos/repoName 2024-05-31 08:44:48.459 [info] Running discovery for pytest using the new test adapter. 2024-05-31 08:44:48.459 [info] Discover tests for workspace name: repoName - uri: /Users/user.name/repos/repoName 2024-05-31 08:44:48.459 [info] Running discovery for pytest using the new test adapter. 2024-05-31 08:44:48.459 [error] Test discovery already in progress, not starting a new one. 2024-05-31 08:44:48.460 [info] All environment variables set for pytest discovery: {"COMMAND_MODE":"unix2003","HOME":"/Users/user.name","LOGNAME":"user.name","MallocNanoZone":"0","OLDPWD":"/","ORIGINAL_XDG_CURRENT_DESKTOP":"undefined","PWD":"/","SHELL":"/bin/zsh","SHLVL":"0","SSH_AUTH_SOCK":"/private/tmp/com.apple.launchd.k1JVAzLpWS/Listeners","TMPDIR":"/var/folders/y6/f9w1yy2s7ld7lm4y2mq_6f6h0000gq/T/","USER":"user.name","VSCODE_AMD_ENTRYPOINT":"vs/workbench/api/node/extensionHostProcess","VSCODE_CODE_CACHE_PATH":"/Users/user.name/Library/Application Support/Code/CachedData/dc96b837cf6bb4af9cd736aa3af08cf8279f7685","VSCODE_CRASH_REPORTER_PROCESS_TYPE":"extensionHost","VSCODE_CWD":"/","VSCODE_HANDLES_UNCAUGHT_ERRORS":"true","VSCODE_IPC_HOOK":"/Users/user.name/Library/Application Support/Code/1.89-main.sock","VSCODE_NLS_CONFIG":"{\"locale\":\"en-us\",\"osLocale\":\"en-us\",\"availableLanguages\":{},\"_languagePackSupport\":true}","VSCODE_PID":"33085","XPC_FLAGS":"0x0","XPC_SERVICE_NAME":"application.com.microsoft.VSCode.47953812.47953818","_":"/Applications/Visual Studio Code.app/Contents/MacOS/Electron","__CFBundleIdentifier":"com.microsoft.VSCode","__CF_USER_TEXT_ENCODING":"0x1F7:0x0:0x0","ELECTRON_RUN_AS_NODE":"1","VSCODE_L10N_BUNDLE_LOCATION":"","APPLICATION_INSIGHTS_NO_DIAGNOSTIC_CHANNEL":"1","GOMODCACHE":"/Users/user.name/go/pkg/mod","GOPATH":"/Users/user.name/go","GOPROXY":"https://proxy.golang.org,direct","PATH":"/Library/Frameworks/Python.framework/Versions/3.11/bin:/usr/local/bin:/System/Cryptexes/App/usr/bin:/usr/bin:/bin:/usr/sbin:/sbin:/usr/local/go/bin:/var/run/com.apple.security.cryptexd/codex.system/bootstrap/usr/local/bin:/var/run/com.apple.security.cryptexd/codex.system/bootstrap/usr/bin:/var/run/com.apple.security.cryptexd/codex.system/bootstrap/usr/appleinternal/bin","PYTHONPATH":"/Users/user.name/.vscode/extensions/ms-python.python-2024.6.0/python_files","TEST_RUN_PIPE":"/var/folders/y6/f9w1yy2s7ld7lm4y2mq_6f6h0000gq/T/python-test-discovery-b0e540957f4429665d38.sock"} 2024-05-31 08:44:48.470 [info] > . ~/repos/repoName/.venv/bin/activate && echo 'e8b39361-0157-4923-80e1-22d70d46dee6' && python ~/.vscode/extensions/ms-python.python-2024.6.0/python_files/printEnvVariables.py 2024-05-31 08:44:48.471 [info] shell: bash 2024-05-31 08:44:48.531 [info] > ~/repos/repoName/.venv/bin/python -m pytest -p vscode_pytest --collect-only "-o 'pythonpath=~/repos/repoName/presets" 2024-05-31 08:44:48.531 [info] cwd: ~/repos/repoName 2024-05-31 08:44:48.889 [info] ============================= test session starts ============================== platform darwin -- Python 3.11.9, pytest-7.4.4, pluggy-1.5.0 2024-05-31 08:44:48.891 [info] rootdir: /Users/user.name/repos/repoName configfile: pytest.ini plugins: mock-3.11.1 2024-05-31 08:44:49.222 [info] collected 28 items / 3 errors ... ```

Extension version: 2024.6.0 VS Code version: Code 1.89.1 (Universal) (dc96b837cf6bb4af9cd736aa3af08cf8279f7685, 2024-05-07T05:14:24.611Z) OS version: Darwin x64 22.6.0 Modes:

User Settings

``` Multiroot scenario, following user settings may not apply: envFile: "" languageServer: "Pylance" testing • pytestEnabled: true ```

Installed Extensions |Extension Name|Extension Id|Version| |---|---|---| |C/C++|ms-vscode.cpptools|1.20.5| |C/C++ Extension Pack|ms-vscode.cpptools-extension-pack|1.3.0| |C/C++ Themes|ms-vscode.cpptools-themes|2.0.0| |CMake|twxs.cmake|0.0.17| |CMake Tools|ms-vscode.cmake-tools|1.18.41| |Go|golang.go|0.41.4| |HTML CSS Support|ecmel.vscode-html-css|2.0.9| |JavaScript Debugger|ms-vscode.js-debug|1.89.0| |JavaScript Debugger Companion Extension|ms-vscode.js-debug-companion|1.1.2| |Pylance|ms-python.vscode-pylance|2024.5.1| |Python|ms-python.python|2024.6.0| |Python Debugger|ms-python.debugpy|2024.6.0| |Table Visualizer for JavaScript Profiles|ms-vscode.vscode-js-profile-table|1.0.9|
System Info |Item|Value| |---|---| |CPUs|Intel(R) Core(TM) i9-9880H CPU @ 2.30GHz (16 x 2300)| |GPU Status|2d_canvas: enabled
canvas_oop_rasterization: disabled_off
direct_rendering_display_compositor: disabled_off_ok
gpu_compositing: enabled
multiple_raster_threads: enabled_on
opengl: enabled_on
rasterization: enabled
raw_draw: disabled_off_ok
skia_graphite: disabled_off
video_decode: enabled
video_encode: enabled
webgl: enabled
webgl2: enabled
webgpu: enabled| |Load (avg)|2, 2, 2| |Memory (System)|64.00GB (26.65GB free)| |Process Argv|--crash-reporter-id fc075b09-bfd0-4e88-b48d-c483d3b60a59| |Screen Reader|no| |VM|0%|
A/B Experiments ``` vsliv368cf:30146710 vspor879:30202332 vspor708:30202333 vspor363:30204092 vswsl492:30256859 vscoreces:30445986 vscod805:30301674 binariesv615:30325510 vsaa593:30376534 py29gd2263:31024239 c4g48928:30535728 azure-dev_surveyone:30548225 962ge761:30959799 pythongtdpath:30769146 welcomedialog:30910333 pythonidxpt:30866567 pythonnoceb:30805159 asynctok:30898717 pythontestfixt:30902429 pythonregdiag2:30936856 pythonmypyd1:30879173 pythoncet0:30885854 2e7ec940:31000449 pythontbext0:30879054 accentitlementsc:30995553 dsvsc016:30899300 dsvsc017:30899301 dsvsc018:30899302 cppperfnew:31000557 dsvsc020:30976470 pythonait:31006305 jchc7451:31062275 chatpanelt:31048053 dsvsc021:30996838 jg8ic977:31013176 pythoncenvpt:31062603 a69g1124:31058053 dvdeprecation:31061160 pythonprt:31056678 dwnewjupyter:31046869 26j00206:31048877 ```
karthiknadig commented 1 month ago

@broganrossae We only do that in the logs, actual execution uses full path that you provide. Would it be possible to share the error that you see when discovery fails?

broganrossae commented 1 month ago

I'm working on repositories with non-standard layouts, and because of that I need to set the PYTHONPATH env var to help pytest find the correct modules. Therefore the errors I get are ModuleNotFoundErrors. In a separate shell I ran the example pytest commands (in the same working dir). One with ~ and one with the absolute path to user's home directory. When using ~ they failed, but the absolute path worked, which is what makes me think the plugin isn't using ~ only in logs.

Error stack example with truncated/redacted stuff

``` 2024-06-03 09:11:07.154 [info] ============================= test session starts ============================== platform darwin -- Python 3.11.9, pytest-7.4.4, pluggy-1.5.0 2024-06-03 09:11:07.162 [info] rootdir: /Users/user.name/repos/repoName configfile: pytest.ini plugins: mock-3.11.1 2024-06-03 09:11:07.506 [info] collected 28 items / 4 errors 2024-06-03 09:11:07.533 [error] pytest test discovery error for workspace: /Users/user.name/repos/repoName _pytest.nodes.Collector.CollectError: ImportError while importing test module '/Users/user.name/repos/repoName/tests/unit/test_another.py'. Hint: make sure your test modules/packages have valid Python names. Traceback: /Library/Frameworks/Python.framework/Versions/3.11/lib/python3.11/importlib/__init__.py:126: in import_module return _bootstrap._gcd_import(name[level:], package, level) tests/unit/test_EXT03_05.py:11: in import EXT03_05_prepareSendFiles as preset presets/EXT03_05_prepareSendFiles.py:9: in import EXT03_00_config as config E ModuleNotFoundError: No module named 'EXT03_00_config' Check Python Test Logs for more details. 2024-06-03 09:11:07.535 [info] =========================== short test summary info ============================ ERROR tests/unit/test_another.py !!!!!!!!!!!!!!!!!!! Interrupted: 4 errors during collection !!!!!!!!!!!!!!!!!!!! ==================== 28 tests collected, 4 errors in 0.38s ===================== 2024-06-03 09:11:07.574 [error] Subprocess exited unsuccessfully with exit code 2 and signal null on workspace /Users/user.name/repos/repoName. 2024-06-03 09:11:07.574 [error] Subprocess exited unsuccessfully with exit code 2 and signal null on workspace /Users/user.name/repos/repoName. Creating and sending error discovery payload 2024-06-03 09:11:07.574 [error] pytest test discovery error for workspace: /Users/user.name/repos/repoName The python test process was terminated before it could exit on its own, the process errored with: Code: 2, Signal: null for workspace /Users/user.name/repos/repoName ```

eleanorjboyd commented 1 week ago

Are you able to send a repo where you are seeing this so I can try it myself? Also which path did you edit in this " One with ~ and one with the absolute path to user's home directory. When using ~ they failed, but the absolute path worked," that caused the change? I am not seeing the ~ in the logs which is why I wanted to clarify. Thanks!

broganrossae commented 6 days ago

I haven't been able to reproduce the exact same behavior, but I did notice that this repo cannot find the module, and it looks like it's the " around the pytest command line arguments.

If, in the terminal, I run

python -m pytest --collect-only "-o pythonpath=/Users/user.name/extra/path"

I get ModuleNotFound errors. However, if I run

python -m pytest --collect-only -o pythonpath=/Users/user.name/extra/path

Modules are found. So maybe that's what was causing my issue.

eleanorjboyd commented 6 days ago

Thanks for following up and glad to hear you are not experiencing the issue anymore! The solution you thought of makes sense- each arg needs to be its own string so that could likely be the cause. Thanks

broganrossae commented 6 days ago

@eleanorjboyd Not sure why you closed this, unless you opened another one for the quotes in command line arguments.

eleanorjboyd commented 6 days ago

I think if you put your args like this then it should work since you need to have each arg in a separate string:

"python.testing.pytestArgs: [
      "-o",
"pythonpath=/User/user.name/repos/extra/path"
    ]
broganrossae commented 6 days ago

Oh, I see. Ok. I guess I was confused by the documentation: Arguments passed in. Each argument is a separate item in the array. I interpreted that meaning each argument should be one item in the array. Guess, that's on me.

eleanorjboyd commented 6 days ago

ah gotcha- I will note it down to make the documentation better! Sorry for the confusion!