pytest-dev / pytest-forked

extracted --boxed from pytest-xdist to ensure backward compat
MIT License
62 stars 21 forks source link

[tests] test_xfail fails against Pytest 8 #89

Closed stanislavlevin closed 6 months ago

stanislavlevin commented 6 months ago
============================= test session starts ==============================
platform linux -- Python 3.12.1, pytest-8.0.0, pluggy-1.4.0
rootdir: /usr/src/RPM/BUILD/pytest-forked
configfile: tox.ini
plugins: forked-1.6.1.dev4+gd9d05e2
collected 10 items

testing/test_boxed.py ...xx.                                             [ 60%]
testing/test_xfail_behavior.py ...F                                      [100%]

=================================== FAILURES ===================================
_________________________ test_xfail[non-strict xpass] _________________________

is_crashing = False, is_strict = False
testdir = <Testdir local('/usr/src/tmp/pytest-of-builder/pytest-4/test_xfail3')>

    @pytest.mark.parametrize(
        ("is_crashing", "is_strict"),
        (
            pytest.param(True, True, id="strict xfail"),
            pytest.param(False, True, id="strict xpass"),
            pytest.param(True, False, id="non-strict xfail"),
            pytest.param(False, False, id="non-strict xpass"),
        ),
    )
    def test_xfail(is_crashing, is_strict, testdir):
        """Test xfail/xpass/strict permutations."""

...

        pytest_run_result = testdir.runpytest(test_module, "-ra")
>       pytest_run_result.stdout.fnmatch_lines(expected_lines)
E       Failed: fnmatch: '*==== test session starts ====*'
E       Failed: fnmatch: '*==== test session starts ====*'
E          with: '============================= test session starts =============================='
E       nomatch: 'plugins: forked*'
E           and: 'platform linux -- Python 3.12.1, pytest-8.0.0, pluggy-1.4.0'
E           and: 'rootdir: /usr/src/tmp/pytest-of-builder/pytest-4/test_xfail3'
E       fnmatch: 'plugins: forked*'
E          with: 'plugins: forked-1.6.1.dev4+gd9d05e2'
E       exact match: 'collected 1 item'
E       nomatch: 'test_xfail.py X*'
E           and: ''
E       fnmatch: 'test_xfail.py X*'
E          with: 'test_xfail.py X                                                          [100%]'
E       nomatch: '*==== short test summary info ====*'
E           and: ''
E           and: '=================================== XPASSES ===================================='
E       fnmatch: '*==== short test summary info ====*'
E          with: '=========================== short test summary info ============================'
E       nomatch: 'XPASS test_xfail.py::test_function The process gets terminated'
E           and: 'XPASS test_xfail.py::test_function - The process gets terminated'
E           and: '============================== 1 xpassed in 0.03s =============================='
E       remains unmatched: 'XPASS test_xfail.py::test_function The process gets terminated'

/usr/src/RPM/BUILD/pytest-forked/testing/test_xfail_behavior.py:121: Failed
----------------------------- Captured stdout call -----------------------------
============================= test session starts ==============================
platform linux -- Python 3.12.1, pytest-8.0.0, pluggy-1.4.0
rootdir: /usr/src/tmp/pytest-of-builder/pytest-4/test_xfail3
plugins: forked-1.6.1.dev4+gd9d05e2
collected 1 item

test_xfail.py X                                                          [100%]

=================================== XPASSES ====================================
=========================== short test summary info ============================
XPASS test_xfail.py::test_function - The process gets terminated
============================== 1 xpassed in 0.03s ==============================

https://docs.pytest.org/en/stable/changelog.html#pytest-8-0-0rc2-2024-01-17

For xpasses, add - in summary between test name and reason, to match how xfail is displayed.