Test is not marked as xfailed when pytest.xfail() called after a failure.

[harmin-parra]

Issue https://github.com/okken/pytest-check/issues/134 does have a fix.

Sure, it isn't a trivial solution, but it is possible to fix it.

In the pytest_runtest_makereport you need to filter the test outcome report. Exceptions during execution can be retrieved with call.excinfo._excinfo.

The fix can resemble something like this:

def pytest_runtest_makereport(item, call):
    outcome = yield
    report = outcome.get_result()

    num_failures = check_log._num_failures
    failures = check_log.get_failures()
    check_log.clear_failures()

    if report.when == 'call':
        # Don't modify test outcome status for xfailed/skipped tests provoked by pytest.fail or pytest.skip calls
        if hasattr(call, 'excinfo') \
                and call.excinfo is not None \
                and call.excinfo.typename in ('Failed', 'Skipped') \
                and hasattr(call.excinfo.value, "msg")
            return

        # Don't modify test outcome status for xfailed tests provoked by pytest.xfail calls
        if hasattr(call, 'excinfo') \
                and call.excinfo is not None \
                and call.excinfo.typename not in ('Failed', 'Skipped')\
                and hasattr(call.excinfo, '_excinfo') \
                and call.excinfo._excinfo is not None \
                and isinstance(call.excinfo._excinfo, tuple) \
                and len(call.excinfo._excinfo) > 1:
            return

    if failures:
        if item._store[xfailed_key]:
            report.outcome = "skipped"
            report.wasxfail = item._store[xfailed_key].reason
        else:

            summary = f"Failed Checks: {num_failures}"
            longrepr = ["\n".join(failures)]
            longrepr.append("-" * 60)
            longrepr.append(summary)

            if report.longrepr:
                longrepr.append("-" * 60)
                longrepr.append(report.longreprtext)
                report.longrepr = "\n".join(longrepr)
            else:
                report.longrepr = "\n".join(longrepr)
            report.outcome = "failed"
            try:
                raise AssertionError(report.longrepr)
            except AssertionError:
                excinfo = ExceptionInfo.from_current()
            call.excinfo = excinfo

[okken]

Interesting. Do you want to work on this?

[harmin-parra]

Sure, I can take a look at that