pytest-dev / pytest

The pytest framework makes it easy to write small tests, yet scales to support complex functional testing
https://pytest.org
MIT License
11.96k stars 2.66k forks source link

Skipping teardown with KeyboardInterrupt #4517

Open dpatel19 opened 5 years ago

dpatel19 commented 5 years ago

I think there's bug regarding running pytest_runtest_teardown in the presence of a KeyboardInterrupt: Expected (no KeyboardInterrupt):

=========================================================== test session starts ============================================================
platform linux -- Python 3.6.5, pytest-4.0.0, py-1.5.4, pluggy-0.8.0 -- /usr/bin/python3
cachedir: .pytest_cache
rootdir: path, inifile: pytest.ini
plugins: timeout-1.3.1
collected 1 item                                                                                                                           

test_teardown_stack.py::test_me setting up session_fixture
done session_fixture set up
setting up my_fixture
setting up fixture_dep
done fixture_dep set up
done my_fixture set up
in test
PASSEDruntest_teardown, pre teardown
tearing down my_fixture
tearing down fixture_dep
tearing down session_fixture
runtest_teardown, post teardown
sessionfinish

========================================================= 1 passed in 0.02 seconds =========================================================

Observed (with KeyboardInterrupt):

=========================================================== test session starts ============================================================
platform linux -- Python 3.6.5, pytest-4.0.0, py-1.5.4, pluggy-0.8.0 -- /usr/bin/python3
cachedir: .pytest_cache
rootdir: path, inifile: pytest.ini
plugins: timeout-1.3.1
collected 1 item                                                                                                                           

test_teardown_stack.py::test_me setting up session_fixture
done session_fixture set up
setting up my_fixture
setting up fixture_dep
done fixture_dep set up
done my_fixture set up
in test
tearing down my_fixture
tearing down fixture_dep
tearing down session_fixture
sessionfinish

!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!! KeyboardInterrupt !!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!
path/test_teardown_stack.py:35: KeyboardInterrupt
(to show a full traceback on KeyboardInterrupt use --fulltrace)
======================================================= no tests ran in 0.29 seconds =======================================================

Note the missing 'runtest_teardown, pre teardown' and 'runtest_teardown, post teardown' in the observed output.

Second, there seems to be bug regarding running multiple finalizers in the presence of a KeyboardInterrupt AND an exception raising in one of the finalizers.

Expected (no KeyboardInterrupt):

=========================================================== test session starts ============================================================
platform linux -- Python 3.6.5, pytest-4.0.0, py-1.5.4, pluggy-0.8.0 -- /usr/bin/python3
cachedir: .pytest_cache
rootdir: path, inifile: pytest.ini
plugins: timeout-1.3.1
collected 1 item                                                                                                                           

test_teardown_stack.py::test_me setting up session_fixture
done session_fixture set up
setting up my_fixture
setting up fixture_dep
done fixture_dep set up
done my_fixture set up
in test
PASSEDruntest_teardown, pre teardown
tearing down my_fixture
tearing down fixture_dep
tearing down session_fixture
runtest_teardown, post teardown

test_teardown_stack.py::test_me ERRORsessionfinish

================================================================== ERRORS ==================================================================
_______________________________________________________ ERROR at teardown of test_me _______________________________________________________

>   request.addfinalizer(lambda: fin())

test_teardown_stack.py:9: 
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 

    def fin():
        print('tearing down fixture_dep')
>       raise Exception('error in fixture_dep')
E       Exception: error in fixture_dep

test_teardown_stack.py:7: Exception
==================================================== 1 passed, 1 error in 0.05 seconds =====================================================

Observed (with KeyboardInterrupt):

=========================================================== test session starts ============================================================
platform linux -- Python 3.6.5, pytest-4.0.0, py-1.5.4, pluggy-0.8.0 -- /usr/bin/python3
cachedir: .pytest_cache
rootdir: path/, inifile: pytest.ini
plugins: timeout-1.3.1
collected 1 item                                                                                                                           

test_teardown_stack.py::test_me setting up session_fixture
done session_fixture set up
setting up my_fixture
setting up fixture_dep
done fixture_dep set up
done my_fixture set up
in test
tearing down my_fixture
tearing down fixture_dep

!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!! KeyboardInterrupt !!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!
path/test_teardown_stack.py:35: KeyboardInterrupt
(to show a full traceback on KeyboardInterrupt use --fulltrace)
Traceback (most recent call last):
...
    raise Exception('error in fixture_dep')
Exception: error in fixture_dep

Note the missing pytest_runtest_teardown pre and post, session fixture teardown, and sessionfinish. In both these cases, I expect teardown to proceed the same (runtest_teardown.pre, fixture teardown, runtest_teardown.post, session fixture teardown, sessionfinish), regardless of a KeyboardInterrupt. Am I missing an option to force teardown, even with KeyboardInterrupt?

Source: conftest.py:

import pytest

@pytest.hookimpl(hookwrapper=True)
def pytest_runtest_teardown(item, nextitem):
    print('runtest_teardown, pre teardown')
    yield
    print('runtest_teardown, post teardown')

@pytest.hookimpl(trylast=True)
def pytest_sessionfinish(session, exitstatus):
    print('sessionfinish')

test_teardown_stack.py:

import pytest

def fixture_dep(request):
    print('setting up fixture_dep')
    def fin():
        print('tearing down fixture_dep')
        # Uncomment for second test case
        # raise Exception('error in fixture_dep')

    request.addfinalizer(lambda: fin())
    print('done fixture_dep set up')

@pytest.fixture()
def my_fixture(request):
    print('setting up my_fixture')
    def fin():
        print('tearing down my_fixture')

    fixture_dep(request)
    request.addfinalizer(lambda: fin())
    print('done my_fixture set up')

@pytest.fixture(scope='session')
def session_fixture(request):
    print('setting up session_fixture')
    def fin():
        print('tearing down session_fixture')

    request.addfinalizer(lambda: fin())
    print('done session_fixture set up')

def test_me(session_fixture, my_fixture):
    print('in test')
    # Comment out for expected case
    raise KeyboardInterrupt()
blueyed commented 4 years ago

Just for reference: related to the pytest_sessionfinish issue, at least with regard to touching code there: https://github.com/pytest-dev/pytest/pull/6660