No visual indication if pytest `teardown` fails

[dmakhno]

Type: Bug

Behaviour

Expected vs. Actual

In Test Results and everywhere test run if has issues during tear down is green.

Consider on single run the might be couple tests. And teardown e.g. module level might group some of them. The one-size-fits all might be adding node for failed teardowns (fixtures). Or fail run, if there is any error, according to https://docs.pytest.org/en/stable/reference/exit-codes.html.

Imho, particular test should stay green, if all its teardown passes. Thus if module teardown fail, related module tests with uses this fixture should fail... but not test in the same module if they don't use this fixture.

References:

• 21722 (seems like this, steps are the same, I added only class level with group exception)

• 21625

Steps to reproduce:

import pytest

@pytest.fixture
def broken_release():
    yield
    raise Exception('cannot teardown')

@pytest.fixture(scope='class')
def broken_release_cls():
    yield
    raise Exception('cannot teardown cls')

def test_resource(broken_release):
    pass

class TestResource:
    def test_class_resource(self, broken_release, broken_release_cls):
        pass

Diagnostic data

• Python version (& distribution if applicable, e.g. Anaconda): 3.12.1
• Type of virtual environment used (e.g. conda, venv, virtualenv, etc.): Global
• Value of the python.languageServer setting: Default

Output for Python in the Output panel (View→Output, change the drop-down the upper-right of the Output panel to Python)

``` XXX ```

User Settings

``` languageServer: "Pylance" testing • cwd: "" • pytestEnabled: true ```

Extension version: 2023.22.1 VS Code version: Code 1.85.2 (Universal) (8b3775030ed1a69b13e4f4c628c612102e30a681, 2024-01-18T06:40:32.531Z) OS version: Darwin arm64 23.2.0 Modes: Connection to 'SSH: switch' could not be established Canceled

System Info

|Item|Value| |---|---| |CPUs|Apple M2 Max (12 x 24)| |GPU Status|2d_canvas: enabled
canvas_oop_rasterization: enabled_on
direct_rendering_display_compositor: disabled_off_ok
gpu_compositing: enabled
multiple_raster_threads: enabled_on
opengl: enabled_on
rasterization: enabled
raw_draw: disabled_off_ok
video_decode: enabled
video_encode: enabled
vulkan: disabled_off
webgl: enabled
webgl2: enabled
webgpu: enabled| |Load (avg)|2, 2, 2| |Memory (System)|96.00GB (26.13GB free)| |Process Argv|--crash-reporter-id 0ddd1806-9bb8-4a88-8fc3-dfe586936e48| |Screen Reader|no| |VM|0%| Connection to 'SSH: switch' could not be established Canceled

A/B Experiments

``` vsliv368:30146709 vspor879:30202332 vspor708:30202333 vspor363:30204092 vswsl492:30256859 vscod805:30301674 binariesv615:30325510 vsaa593:30376534 py29gd2263:30899288 vsclangdc:30486549 c4g48928:30535728 azure-dev_surveyone:30548225 2i9eh265:30646982 962ge761:30933248 pythongtdpath:30769146 welcomedialog:30910333 pythonidxpt:30866567 pythonnoceb:30805159 asynctok:30898717 pythontestfixt:30902429 pythonregdiag2:30936856 pyreplss1:30897532 pythonmypyd1:30879173 pythoncet0:30885854 pythontbext0:30879054 dsvsc016:30899300 dsvsc017:30899301 dsvsc018:30899302 ```

[dmakhno]

@tekumara, do you see resolved #21722 ?

[eleanorjboyd]

thanks for the bug- I will investigate

[dmakhno]

@eleanorjboyd, were you able to see and repro the issue?

If it matters for the motivation part:

• I have a heavy teardown by SUT nature, and if teardown fails (environment validation)... there is no sense to start next test, since this test did not clean after itself.
• And... with existing confusion, it is easy to miss result - and starting to dig wrong (following after) test.

[tekumara]

For me exceptions before yield make the test go red but after yield it's green.

[brucepang]

also ran into this issue and would like to have it supported.