search

microsoft / vscode-python

Python extension for Visual Studio Code
https://aka.ms/pvsc-marketplace
MIT License
4.32k stars 1.18k forks source link

Pytest code coverage does not show up #24241

Open loewexy opened 3 weeks ago

loewexy commented 3 weeks ago

Type: Bug

Behaviour

Clicking on the Run Tests with Coverage button does not have any effect. No coverage data is generated.

As you can see in the output no option enabling coverage is also passed to pytest.

I use VScode with WSL. I tested the same on a native Linux Computer and it worked as expected.

Steps to reproduce:

  1. Setup any simple project with a pytest test.
  2. Click on Run test with Coverage button.

Diagnostic data

Output for Python in the Output panel (View→Output, change the drop-down the upper-right of the Output panel to Python)

``` 2024-10-04 10:27:29.384 [info] Default formatter is set to null for workspace /home/user/dev/test 2024-10-04 10:27:30.088 [info] Python interpreter path: ./.venv/bin/python 2024-10-04 10:27:30.533 [info] > ~/.pyenv/bin/pyenv which python 2024-10-04 10:27:30.533 [info] cwd: . 2024-10-04 10:27:32.070 [info] Discover tests for workspace name: test - uri: /home/user/dev/test 2024-10-04 10:27:32.105 [info] > pixi --version 2024-10-04 10:27:32.121 [info] > . ./.venv/bin/activate && echo 'e8b39361-0157-4923-80e1-22d70d46dee6' && python ~/.vscode-server/extensions/ms-python.python-2024.16.0/python_files/printEnvVariables.py 2024-10-04 10:27:32.121 [info] shell: bash 2024-10-04 10:27:32.159 [info] > ./.venv/bin/python ~/.vscode-server/extensions/ms-python.python-2024.16.0/python_files/testing_tools/run_adapter.py discover pytest -- --rootdir . -s --cache-clear tests 2024-10-04 10:27:32.159 [info] cwd: . 2024-10-04 10:27:33.744 [info] > ./.venv/bin/python -m pytest --rootdir . --override-ini junit_family=xunit1 --junit-xml=/tmp/tmp-7959CmUhSOd99eL3.xml 2024-10-04 10:27:33.744 [info] cwd: . 2024-10-04 10:27:41.632 [info] Starting Pylance language server. ```

Extension version: 2024.16.0 VS Code version: Code 1.94.0 (d78a74bcdfad14d5d3b1b782f87255d802b57511, 2024-10-02T13:08:12.626Z) OS version: Windows_NT x64 10.0.19045 Modes: Remote OS version: Linux x64 5.15.153.1-microsoft-standard-WSL2

User Settings

``` languageServer: "Pylance" testing • pytestArgs: "" • pytestEnabled: true ```

Installed Extensions |Extension Name|Extension Id|Version| |---|---|---| |autoDocstring - Python Docstring Generator|njpwerner.autodocstring|0.6.1| |autopep8|ms-python.autopep8|2024.0.0| |Better Jinja|samuelcolvin.jinjahtml|0.20.0| |Even Better TOML|tamasfe.even-better-toml|0.19.2| |Flake8|ms-python.flake8|2023.10.0| |Git Blame|waderyan.gitblame|11.1.0| |Git Graph|mhutchie.git-graph|1.30.0| |GitLab Workflow|GitLab.gitlab-workflow|5.14.0| |isort|ms-python.isort|2023.10.1| |JavaScript Debugger|ms-vscode.js-debug|1.94.0| |Mypy Type Checker|ms-python.mypy-type-checker|2024.0.0| |Pylance|ms-python.vscode-pylance|2024.9.2| |Python|ms-python.python|2024.16.0| |Python Debugger|ms-python.debugpy|2024.10.0| |REST Client|humao.rest-client|0.25.1| |Table Visualizer for JavaScript Profiles|ms-vscode.vscode-js-profile-table|1.0.10| |YAML|redhat.vscode-yaml|1.15.0|
System Info |Item|Value| |---|---| |CPUs|11th Gen Intel(R) Core(TM) i5-1145G7 @ 2.60GHz (8 x 1498)| |GPU Status|2d_canvas: enabled
canvas_oop_rasterization: enabled_on
direct_rendering_display_compositor: disabled_off_ok
gpu_compositing: enabled
multiple_raster_threads: enabled_on
opengl: enabled_on
rasterization: enabled
raw_draw: disabled_off_ok
skia_graphite: disabled_off
video_decode: enabled
video_encode: enabled
vulkan: disabled_off
webgl: enabled
webgl2: enabled
webgpu: enabled
webnn: disabled_off| |Load (avg)|undefined| |Memory (System)|15.73GB (4.64GB free)| |Process Argv|--crash-reporter-id 4f92d2c0-8402-4937-a1b9-ca00fe63248a| |Screen Reader|no| |VM|40%| |Item|Value| |---|---| |Remote|WSL: ubuntu-22-04| |OS|Linux x64 5.15.153.1-microsoft-standard-WSL2| |CPUs|11th Gen Intel(R) Core(TM) i5-1145G7 @ 2.60GHz (8 x 0)| |Memory (System)|7.63GB (6.02GB free)| |VM|0%|
A/B Experiments ``` vsliv368:30146709 vspor879:30202332 vspor708:30202333 vspor363:30204092 vswsl492:30256859 vscod805:30301674 binariesv615:30325510 vsaa593cf:30376535 py29gd2263:31024239 c4g48928:30535728 azure-dev_surveyone:30548225 962ge761:30959799 pythongtdpath:30769146 welcomedialog:30910333 pythonnoceb:30805159 asynctok:30898717 pythonmypyd1:30879173 h48ei257:31000450 pythontbext0:30879054 accentitlementst:30995554 dsvsc016:30899300 dsvsc017:30899301 dsvsc018:30899302 cppperfnew:31000557 dsvsc020:30976470 pythonait:31006305 dsvsc021:30996838 jg8ic977:31013176 a69g1124:31058053 dvdeprecation:31068756 dwnewjupyter:31046869 2f103344:31071589 impr_priority:31102340 nativerepl2:31139839 refactort:31108082 pythonrstrctxt:31112756 flightc:31134773 wkspc-onlycs-t:31132770 nativeloc2:31134642 wkspc-ranged-t:31151552 cf971741:31144450 autoexpandse:31146404 iacca2:31150323 notype1cf:31151524 cc771715:31146322 ```
loewexy commented 3 weeks ago

It seems like the extensions does not see that pytest-cov is installed in that case. But I do not know how I can check what the reason for this behaviour is.

Running pytest with the --cov=. option directly on the cli works without problems.

fbuessen commented 3 weeks ago

I can confirm this behavior (tests run, but no coverage is generated). I'm also seeing the same behavior when running VSCode+Python natively on Windows. @eleanorjboyd

falkben commented 3 weeks ago

Also seeing this behavior on Mac. Using conda for virtual environment

eleanorjboyd commented 3 weeks ago

Hi! From your logs it looks like you aren't on the testing rewrite which is required. Could you update to our rewrite and try again? You can do so adding this setting to your users settings.json "python.experiments.optInto": ["pythonTestAdapter"]. Then you can confirm its enabled by finding Experiment 'pythonTestAdapter' is active in your python logs. If you are unable to see the log for the experiment change your log level to trace and it should show up. To set to trace use theDeveloper: set log level command in the command palette. Thanks

fbuessen commented 3 weeks ago

Hi! From your logs it looks like you aren't on the testing rewrite which is required. Could you update to our rewrite and try again? You can do so adding this setting to your users settings.json "python.experiments.optInto": ["pythonTestAdapter"].

Ah, the pythonTestAdapter experiment fixes the issue. Thanks!

eleanorjboyd commented 3 weeks ago

For clarity I have created a PR so the button is only surfaced on the rewrite where coverage is supported: https://github.com/microsoft/vscode-python/pull/24249.

loewexy commented 3 weeks ago

I will check tomorrow if this also fixes the issue for me.

Did I miss this information about the required experiment in the Docs? The release notes and also the Documentation do not mention that the setting is required. And why did it work on my Linux machine out of the box?

Hiding the button is one way to handle it, if this is clearly documented I do not see an issue. The other option would be to present the user with a hint how the feature can be enabled. For me that would have been the best options, otherwise I would have opened an issue that the button is missing.

eleanorjboyd commented 3 weeks ago

yes we will add a doc update to clarify

cc @cwebster-99

loewexy commented 3 weeks ago

I can confirm that the experiment also fixes the issue for me.

I can now also explain why it worked on my linux machine. The experiment has been active there as part of an A/B test. If I disable the A/B tests for the python extensions it also does not work here. Therefore discussions about the platform have not been based on the platform but rather random on the enabled A/B test.

The issue is therefore resolved for me, but I think for closing this issue it is at least required to add this to the documentation, otherwise other users will also be confused.

ivan-toriya commented 3 weeks ago

@eleanorjboyd just curious, is there a timeline when "pythonTestAdapter" will be enabled by default?

eleanorjboyd commented 3 weeks ago

Hi @ivan-toriya, yes we want to get it to default (as it also makes it more maintainable) we are waiting until one final issue with named pipe communication is finished as this causes the new experience to break for some users.

alspadoni commented 3 weeks ago

Even after adding the setting to my json config file I can't see the experiment active in the logs (I changed the log level) nor does the coverage show up. I'm on mac, any ideas?

eleanorjboyd commented 2 days ago

Hi @alspadoni can you try running the command "Python: clear cache and reload window"? (Sorry about the delay as well)

FancyBanana commented 5 hours ago

I've tried adding the optIn option to my settings but I still don't see the coverage button... Here's my settings:

{
    "python.testing.pytestArgs": [
        "core"
    ],
    "python.testing.unittestEnabled": false,
    "python.testing.pytestEnabled": true,
    "python.experiments.optInto": [
        "pythonTestAdapter",
    ],
}
eleanorjboyd commented 4 hours ago

Can you put the experiment opt into setting in your user settings? Then you can confirm its enabled by finding Experiment 'pythonTestAdapter' is active in your python logs. If you are unable to see the log for the experiment change your log level to trace and it should show up. To set to trace use theDeveloper: set log level command in the command palette. Thanks

FancyBanana commented 4 hours ago

@eleanorjboyd It is present in my user settings also. There is nothing present in logs, images attached below. A thing that I've forgot to mention is that I connect into a VM through ssh for this project. Image Image

FancyBanana commented 3 hours ago

Here is my entire python log history:

2024-10-31 16:30:23.815 [warning] Dir "/var/www/html/backend-project/project_project/.pixi/envs" is not watchable (directory does not exist)
2024-10-31 16:30:23.815 [info] Default formatter is set to null for workspace /var/www/html/backend-project/project_project
2024-10-31 16:30:23.815 [info] Starting Environment refresh
2024-10-31 16:30:23.815 [info] Searching for interpreters in posix paths locator
2024-10-31 16:30:23.815 [info] Searching for pyenv environments
2024-10-31 16:30:23.815 [info] Searching for conda environments
2024-10-31 16:30:23.815 [info] Searching for global virtual environments
2024-10-31 16:30:23.815 [info] Searching for custom virtual environments
2024-10-31 16:30:23.815 [info] pyenv is not installed
2024-10-31 16:30:23.815 [info] Finished searching for pyenv environments: 44 milliseconds
2024-10-31 16:30:23.815 [info] Finished searching for custom virtual envs: 42 milliseconds
2024-10-31 16:30:23.815 [info] > conda info --json
2024-10-31 16:30:23.815 [info] > pixi --version
2024-10-31 16:30:23.815 [info] > hatch env show --json
2024-10-31 16:30:23.815 [info] cwd: .
2024-10-31 16:30:23.815 [info] Finished searching for global virtual envs: 81 milliseconds
2024-10-31 16:30:23.815 [info] > /usr/share/anaconda/bin/conda info --json
2024-10-31 16:30:23.815 [info] Send text to terminal: printenv > "/tmp/tmp-10424bo46Vn0R6lPI.txt"
2024-10-31 16:30:23.815 [info] > pyenv which python
2024-10-31 16:30:23.815 [info] cwd: .
2024-10-31 16:30:23.885 [info] Found: /bin/python --> /bin/python3.9
2024-10-31 16:30:24.013 [info] Found: /bin/python3 --> /bin/python3.9
2024-10-31 16:30:24.019 [info] Found: /bin/python3.9 --> /bin/python3.9
2024-10-31 16:30:24.148 [info] > /usr/local/bin/python3.12 -I ~/.vscode-server/extensions/ms-python.python-2024.16.1-linux-x64/python_files/get_output_via_markers.py ~/.vscode-server/extensions/ms-python.python-2024.16.1-linux-x64/python_files/interpreterInfo.py
2024-10-31 16:30:24.154 [info] Found: /usr/bin/python --> /usr/bin/python3.9
2024-10-31 16:30:24.188 [info] Found: /usr/bin/python3 --> /usr/bin/python3.9
2024-10-31 16:30:24.198 [info] Found: /usr/bin/python3.9 --> /usr/bin/python3.9
2024-10-31 16:30:24.204 [info] > /var/www/html/backend-project/env/bin/python -I ~/.vscode-server/extensions/ms-python.python-2024.16.1-linux-x64/python_files/get_output_via_markers.py ~/.vscode-server/extensions/ms-python.python-2024.16.1-linux-x64/python_files/interpreterInfo.py
2024-10-31 16:30:24.216 [info] Found: /usr/local/bin/python3.12 --> /usr/local/bin/python3.12
2024-10-31 16:30:24.223 [info] > /bin/python -I ~/.vscode-server/extensions/ms-python.python-2024.16.1-linux-x64/python_files/get_output_via_markers.py ~/.vscode-server/extensions/ms-python.python-2024.16.1-linux-x64/python_files/interpreterInfo.py
2024-10-31 16:30:24.232 [info] Finished searching for interpreters in posix paths locator: 1639 milliseconds
2024-10-31 16:30:24.232 [info] Environments refresh paths discovered (event): 1639 milliseconds
2024-10-31 16:30:24.232 [info] Environments refresh paths discovered: 1639 milliseconds
2024-10-31 16:30:24.257 [info] > /usr/bin/python -I ~/.vscode-server/extensions/ms-python.python-2024.16.1-linux-x64/python_files/get_output_via_markers.py ~/.vscode-server/extensions/ms-python.python-2024.16.1-linux-x64/python_files/interpreterInfo.py
2024-10-31 16:30:24.261 [info] Python interpreter path: /var/www/html/backend-project/env/bin/python
2024-10-31 16:30:24.325 [info] Environments refresh finished (event): 1732 milliseconds
2024-10-31 16:30:24.341 [info] Environment refresh took 1749 milliseconds
2024-10-31 16:30:24.413 [error] Environment /var/www/html/backend-project/env/bin/python is missing from native locator
2024-10-31 16:30:25.056 [info] Starting Pylance language server.
2024-10-31 16:30:38.342 [debug] Found cached env for /var/www/html/backend-project/env/bin/python
2024-10-31 16:30:38.343 [debug] Found cached env for /var/www/html/backend-project/env/bin/python
2024-10-31 16:30:54.555 [debug] Found cached env for /var/www/html/backend-project/env/bin/python
2024-10-31 16:30:59.000 [debug] Found cached env for /var/www/html/backend-project/env/bin/python
2024-10-31 16:31:07.066 [debug] Found cached env for /var/www/html/backend-project/env/bin/python
2024-10-31 16:31:07.448 [debug] Found cached env for /var/www/html/backend-project/env/bin/python
2024-10-31 16:31:11.076 [debug] Found cached env for /var/www/html/backend-project/env/bin/python
2024-10-31 16:31:11.480 [debug] Found cached env for /var/www/html/backend-project/env/bin/python
2024-10-31 16:31:16.639 [debug] Found cached env for /var/www/html/backend-project/env/bin/python
2024-10-31 16:31:17.185 [debug] Found cached env for /var/www/html/backend-project/env/bin/python
2024-10-31 16:33:04.612 [debug] Testing: Refreshing all test data
2024-10-31 16:33:04.613 [debug] Found cached env for /var/www/html/backend-project/env/bin/python
2024-10-31 16:33:04.613 [debug] Found cached env for /var/www/html/backend-project/env/bin/python
2024-10-31 16:33:04.615 [info] Discover tests for workspace name: project_project - uri: /var/www/html/backend-project/project_project
2024-10-31 16:33:04.615 [debug] Found cached env for /var/www/html/backend-project/env/bin/python
2024-10-31 16:33:04.616 [debug] could not find a pixi interpreter for the interpreter at /var/www/html/backend-project/env/bin/python
2024-10-31 16:33:04.665 [debug] Activation Commands received source /var/www/html/backend-project/env/bin/activate for shell bash, resource /var/www/html/backend-project/project_project and interpreter /var/www/html/backend-project/env/bin/python
2024-10-31 16:33:04.666 [debug] Activating Environment to capture Environment variables, . /var/www/html/backend-project/env/bin/activate && echo 'e8b39361-0157-4923-80e1-22d70d46dee6' && python /home/sow/.vscode-server/extensions/ms-python.python-2024.16.1-linux-x64/python_files/printEnvVariables.py
2024-10-31 16:33:04.666 [info] > . /var/www/html/backend-project/env/bin/activate && echo 'e8b39361-0157-4923-80e1-22d70d46dee6' && python ~/.vscode-server/extensions/ms-python.python-2024.16.1-linux-x64/python_files/printEnvVariables.py
2024-10-31 16:33:04.666 [info] shell: bash
2024-10-31 16:33:04.696 [debug] could not find a pixi interpreter for the interpreter at /var/www/html/backend-project/env/bin/python
2024-10-31 16:33:04.699 [info] > /var/www/html/backend-project/env/bin/python ~/.vscode-server/extensions/ms-python.python-2024.16.1-linux-x64/python_files/testing_tools/run_adapter.py discover pytest -- --rootdir . -s --cache-clear core
2024-10-31 16:33:04.699 [info] cwd: .
2024-10-31 16:33:05.977 [debug] Testing: Resolving item /var/www/html/backend-project/project_project
2024-10-31 16:33:05.978 [debug] Testing: Resolving item /var/www/html/backend-project/project_project/core
2024-10-31 16:33:05.978 [debug] Testing: Resolving item /var/www/html/backend-project/project_project/core/tests
2024-10-31 16:33:07.232 [debug] Found cached env for /var/www/html/backend-project/env/bin/python
2024-10-31 16:33:24.883 [debug] Found cached env for /var/www/html/backend-project/env/bin/python
2024-10-31 16:33:29.635 [debug] Found cached env for /var/www/html/backend-project/env/bin/python
2024-10-31 16:33:37.702 [debug] Found cached env for /var/www/html/backend-project/env/bin/python
eleanorjboyd commented 1 hour ago

yes so this shows you are not on the rewrite 2024-10-31 16:33:04.699 [info] > /var/www/html/backend-project/env/bin/python ~/.vscode-server/extensions/ms-python.python-2024.16.1-linux-x64/python_files/testing_tools/run_adapter.py discover pytest -- --rootdir . -s --cache-clear core. Can you check your VM settings? Check this for the ordering, maybe some setting is override something else https://code.visualstudio.com/docs/getstarted/settings#_settings-precedence