search

microsoft / vscode-python

Python extension for Visual Studio Code
https://aka.ms/pvsc-marketplace
MIT License
4.3k stars 1.18k forks source link

Test Discovery Fails -> sys.path behavior changed? #23587

Closed codeschnabel closed 2 months ago

codeschnabel commented 3 months ago

Type: Bug

Behaviour

When using the Python extension version 2024.8.0 (also broken in 2024.6.0), I cannot discover tests. It works in version 2024.4.1.

Steps to reproduce:

  1. Clone my repo: https://github.com/codeschnabel/vscode-python-test-discovery-problem
  2. Install Python Extension Version 2024.8.0 (I have also installed Pylance, Python Debugger and isort but I don't think this matters).
  3. Shift+Ctrl+P: Python: Configure Tests
  4. Select a test framework/tool to enable: unittest
  5. Select the directory containing the tests: fizzbuzz_test
  6. Select the pattern to identify test files: *_test.py
  7. VSCode Test Explorer fails to discovers Unit Tests ("Unittest Discovery Error... Show output...) image
  8. According to output window the root cause seems to be: ModuleNotFoundError: No module named 'fizzbuzz'

I am not a Python expert, but I think my test file can't be compiled as it doesn't find the fizzbuzz module and I have back-tracked it to a different sys.path starting from Version 2024.6.0.

I have added some diagnostic code to my test file:

image

It seems like the sys.path is set differently in version 2024.8.0 - the workspace folder is not added to the sys.path. In version 2024.4.1 the workspace folder is added to the sys.path.

You can see the different sys paths in the different diagnostic data I provide below:

V2024.8.0:

2024-06-10 19:08:27.322 [info] sys.path: 
['c:\\Users\\stefa\\Repos\\vscode-python-test-discovery-problem\\fizzbuzz_test', 'c:\\Users\\stefa\\.vscode\\extensions\\ms-python.python-2024.8.0\\python_files\\unittestadapter', 'C:\\Users\\stefa\\AppData\\Local\\Programs\\Python\\Python312\\python312.zip', 'C:\\Users\\stefa\\AppData\\Local\\Programs\\Python\\Python312\\DLLs', 'C:\\Users\\stefa\\AppData\\Local\\Programs\\Python\\Python312\\Lib', 'C:\\Users\\stefa\\AppData\\Local\\Programs\\Python\\Python312', 'C:\\Users\\stefa\\AppData\\Local\\Programs\\Python\\Python312\\Lib\\site-packages', 'c:\\Users\\stefa\\.vscode\\extensions\\ms-python.python-2024.8.0\\python_files', 'c:\\Users\\stefa\\.vscode\\extensions\\ms-python.python-2024.8.0\\python_files', 'c:\\Users\\stefa\\.vscode\\extensions\\ms-python.python-2024.8.0\\python_files\\lib\\python']

V2024.4.1:

2024-06-10 19:08:27.322 [info] sys.path: 
sys.path: 
['c:\\Users\\stefa\\Repos\\vscode-python-test-discovery-problem\\fizzbuzz_test', '.', 'c:\\Users\\stefa\\.vscode\\extensions\\ms-python.python-2024.4.1\\python_files\\lib\\python', 'c:\\Users\\stefa\\.vscode\\extensions\\ms-python.python-2024.4.1\\python_files\\unittestadapter', 'c:\\Users\\stefa\\Repos\\vscode-python-test-discovery-problem', 'C:\\Users\\stefa\\AppData\\Local\\Programs\\Python\\Python312\\python312.zip', 'C:\\Users\\stefa\\AppData\\Local\\Programs\\Python\\Python312\\DLLs', 'C:\\Users\\stefa\\AppData\\Local\\Programs\\Python\\Python312\\Lib', 'C:\\Users\\stefa\\AppData\\Local\\Programs\\Python\\Python312', 'C:\\Users\\stefa\\AppData\\Local\\Programs\\Python\\Python312\\Lib\\site-packages', 'c:\\Users\\stefa\\.vscode\\extensions\\ms-python.python-2024.4.1\\python_files', 'c:\\Users\\stefa\\.vscode\\extensions\\ms-python.python-2024.4.1\\python_files', 'c:\\Users\\stefa\\.vscode\\extensions\\ms-python.python-2024.4.1\\python_files\\lib\\python']

Old version 2024.4.1 adds workspace folder to sys.path ('c:\Users\stefa\Repos\vscode-python-test-discovery-problem') --> feels correct for me (?)

Diagnostic data

Output for Python **2024.8.0** in the Output panel (Viewā†’Output, change the drop-down the upper-right of the Output panel to Python)

``` 2024-06-10 19:08:27.192 [info] Discover tests for workspace name: vscode-python-test-discovery-problem - uri: c:\Users\stefa\Repos\vscode-python-test-discovery-problem 2024-06-10 19:08:27.193 [info] Running discovery for unittest using the new test adapter. 2024-06-10 19:08:27.219 [info] Discovering unittest tests for workspace c:\Users\stefa\Repos\vscode-python-test-discovery-problem with arguments: c:\Users\stefa\.vscode\extensions\ms-python.python-2024.8.0\python_files\unittestadapter\discovery.py,--udiscovery,-v,-s,./fizzbuzz_test,-p,*_test.py 2024-06-10 19:08:27.219 [info] > ~\AppData\Local\Programs\Python\Python312\python.exe ~\.vscode\extensions\ms-python.python-2024.8.0\python_files\unittestadapter\discovery.py --udiscovery -v -s ./fizzbuzz_test -p *_test.py 2024-06-10 19:08:27.219 [info] cwd: . 2024-06-10 19:08:27.322 [info] sys.path: ['c:\\Users\\stefa\\Repos\\vscode-python-test-discovery-problem\\fizzbuzz_test', 'c:\\Users\\stefa\\.vscode\\extensions\\ms-python.python-2024.8.0\\python_files\\unittestadapter', 'C:\\Users\\stefa\\AppData\\Local\\Programs\\Python\\Python312\\python312.zip', 'C:\\Users\\stefa\\AppData\\Local\\Programs\\Python\\Python312\\DLLs', 'C:\\Users\\stefa\\AppData\\Local\\Programs\\Python\\Python312\\Lib', 'C:\\Users\\stefa\\AppData\\Local\\Programs\\Python\\Python312', 'C:\\Users\\stefa\\AppData\\Local\\Programs\\Python\\Python312\\Lib\\site-packages', 'c:\\Users\\stefa\\.vscode\\extensions\\ms-python.python-2024.8.0\\python_files', 'c:\\Users\\stefa\\.vscode\\extensions\\ms-python.python-2024.8.0\\python_files', 'c:\\Users\\stefa\\.vscode\\extensions\\ms-python.python-2024.8.0\\python_files\\lib\\python'] 2024-06-10 19:08:27.324 [error] Unittest test discovery error for workspace: c:\Users\stefa\Repos\vscode-python-test-discovery-problem Failed to import test module: fizzbuzzer_test Traceback (most recent call last): File "C:\Users\stefa\AppData\Local\Programs\Python\Python312\Lib\unittest\loader.py", line 396, in _find_test_path module = self._get_module_from_name(name) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "C:\Users\stefa\AppData\Local\Programs\Python\Python312\Lib\unittest\loader.py", line 339, in _get_module_from_name __import__(name) File "c:\Users\stefa\Repos\vscode-python-test-discovery-problem\fizzbuzz_test\fizzbuzzer_test.py", line 7, in from fizzbuzz.fizzbuzzer import FizzBuzzer ModuleNotFoundError: No module named 'fizzbuzz' ```

Extension version: 2024.8.0 VS Code version: Code 1.90.0 (89de5a8d4d6205e5b11647eb6a74844ca23d2573, 2024-06-04T19:33:54.889Z) OS version: Windows_NT x64 10.0.22631 Modes:

User Settings

``` languageServer: "Pylance" testing ā€¢ unittestArgs: "" ā€¢ unittestEnabled: true ```

Installed Extensions |Extension Name|Extension Id|Version| |---|---|---| |Angular Essentials (Version 17)|johnpapa.angular-essentials|17.0.0| |Angular Language Service|Angular.ng-template|18.0.0| |Angular Snippets (Version 16)|johnpapa.Angular2|16.0.1| |Apache Conf|mrmlnc.vscode-apache|1.2.0| |CSS Formatter|aeschli.vscode-css-formatter|1.0.2| |Debugger for Java|vscjava.vscode-java-debug|0.57.0| |Debugger for Microsoft Edge|msjsdiag.debugger-for-edge|1.0.15| |Deno|denoland.vscode-deno|3.37.1| |Docker|ms-azuretools.vscode-docker|1.29.1| |Draw.io Integration|hediet.vscode-drawio|1.6.6| |EditorConfig for VS Code|EditorConfig.EditorConfig|0.16.4| |ESLint|dbaeumer.vscode-eslint|2.4.4| |Extension Pack for Java|vscjava.vscode-java-pack|0.27.0| |Git History|donjayamanne.githistory|0.6.20| |GitHub Copilot|GitHub.copilot|1.172.0| |GitHub Copilot Chat|GitHub.copilot-chat|0.16.0| |IntelliCode|VisualStudioExptTeam.vscodeintellicode|1.3.1| |IntelliCode API Usage Examples|VisualStudioExptTeam.intellicode-api-usage-examples|0.2.8| |isort|ms-python.isort|2023.10.1| |JavaScript Debugger|ms-vscode.js-debug|1.90.0| |JavaScript Debugger Companion Extension|ms-vscode.js-debug-companion|1.1.2| |Jupyter|ms-toolsai.jupyter|2024.5.0| |Jupyter Cell Tags|ms-toolsai.vscode-jupyter-cell-tags|0.1.9| |Jupyter Keymap|ms-toolsai.jupyter-keymap|1.1.2| |Jupyter Notebook Renderers|ms-toolsai.jupyter-renderers|1.0.18| |Jupyter Slide Show|ms-toolsai.vscode-jupyter-slideshow|0.1.6| |Language Support for Java(TM) by Red Hat|redhat.java|1.31.0| |Markdown All in One|yzhang.markdown-all-in-one|3.6.2| |Markdown PDF|yzane.markdown-pdf|1.5.0| |Material Icon Theme|PKief.material-icon-theme|5.3.0| |Maven for Java|vscjava.vscode-maven|0.44.0| |Nx Console|nrwl.angular-console|18.21.4| |Peacock|johnpapa.vscode-peacock|4.2.2| |Prettier - Code formatter|esbenp.prettier-vscode|10.4.0| |Project Manager for Java|vscjava.vscode-java-dependency|0.23.6| |Pylance|ms-python.vscode-pylance|2024.6.1| |Python|ms-python.python|2024.8.0| |Python Debugger|ms-python.debugpy|2024.6.0| |Table Visualizer for JavaScript Profiles|ms-vscode.vscode-js-profile-table|1.0.9| |Test Adapter Converter|ms-vscode.test-adapter-converter|0.1.9| |Test Explorer UI|hbenl.vscode-test-explorer|2.21.1| |Test Runner for Java|vscjava.vscode-java-test|0.41.1| |Winter is Coming Theme|johnpapa.winteriscoming|1.4.4|
System Info |Item|Value| |---|---| |CPUs|Intel(R) Core(TM) i7-10750H CPU @ 2.60GHz (12 x 2592)| |GPU Status|2d_canvas: enabled
canvas_oop_rasterization: enabled_on
direct_rendering_display_compositor: disabled_off_ok
gpu_compositing: enabled
multiple_raster_threads: enabled_on
opengl: enabled_on
rasterization: enabled
raw_draw: disabled_off_ok
skia_graphite: disabled_off
video_decode: enabled
video_encode: enabled
vulkan: disabled_off
webgl: enabled
webgl2: enabled
webgpu: enabled| |Load (avg)|undefined| |Memory (System)|31.84GB (17.92GB free)| |Process Argv|C:\\Temp\\TestDiscoveryProblem --crash-reporter-id 453a5a11-2eec-4a21-8aa8-9040e1259ac4| |Screen Reader|no| |VM|0%|
A/B Experiments ``` vsliv368cf:30146710 vspor879:30202332 vspor708:30202333 vspor363:30204092 vswsl492:30256859 vscoreces:30445986 vscod805:30301674 binariesv615:30325510 vsaa593cf:30376535 py29gd2263:31024239 c4g48928:30535728 azure-dev_surveyone:30548225 962ge761:30959799 pythongtdpath:30769146 welcomedialog:30910333 pythonidxpt:30866567 pythonnoceb:30805159 asynctok:30898717 pythontestfixt:30902429 pythonregdiag2:30936856 pythonmypyd1:30879173 pythoncet0:30885854 h48ei257:31000450 pythontbext0:30879054 accentitlementsc:30995553 dsvsc016:30899300 dsvsc017:30899301 dsvsc018:30899302 cppperfnew:31000557 dsvsc020:30976470 pythonait:31006305 jchc7451:31067544 chatpanelc:31048052 dsvsc021:30996838 0ee40948:31013168 pythoncenvpt:31062603 a69g1124:31058053 dvdeprecation:31068756 pythonprt:31056678 dwnewjupytercf:31046870 26j00206:31048877 ```
eleanorjboyd commented 3 months ago

Will investigate- thanks for the in-depth repro and steps!

daniel-kun commented 3 months ago

@eleanorjboyd Any update on this bug? It's currently blocking our test execution in VS Code and we need to resort to executing it via command line.

eleanorjboyd commented 3 months ago

Hello! Sorry for the delay- do you think this would be the fix? Turns out we removed something unintentionally during a larger overhaul that touches the unittest path variable: https://github.com/microsoft/vscode-python/pull/23712#pullrequestreview-2144519175

daniel-kun commented 3 months ago

@eleanorjboyd I'm afraid I can not answer your question, because I don't know the code of the Extension, but thanks for looking into this! :-)

eleanorjboyd commented 3 months ago

hi @daniel-kun, no worries! Just wasn't sure if you were familiar at all so thanks for getting back to me! This fix is out in insiders so if possible can you switch to use the pre-release version of the python extension and give the scenario a try again? Let me know if it is fixed and if not could you include a fresh set of logs. Thanks!

daniel-kun commented 3 months ago

@eleanorjboyd Fix worked in the Insider version. Thanks a lot šŸ‘