search

numirias / pytest-json-report

🗒️ A pytest plugin to report test results as JSON
MIT License
147 stars 39 forks source link

summary showing xpassed test as passed #39

Closed david-fliguer-ws closed 5 years ago

david-fliguer-ws commented 5 years ago

Hi, first of all kudos for this great plugin

I am running to the following file 

import pytest

def test_one():
    assert True

def test_two():
    assert False

@pytest.mark.skip
def test_three():
    assert True

@pytest.mark.xfail
def test_four():
    assert False

@pytest.mark.xfail
def test_five():
    assert True

But in the report, the summary shows

"summary": {
    "passed": 2,
    "failed": 1,
    "skipped": 1,
    "xfailed": 1,
    "num_tests": 5,
    "duration": 0.08672428131103516
}

seems that test_five is being counted as passed instead of xpassed

Is this a known issue ?

numirias commented 5 years ago

Your report is produced by pytest-json, a different (unmaintained) plugin. My plugin produces the following summary for your test file in the latest version:

"summary": {
    "passed": 1,
    "failed": 1,
    "skipped": 1,
    "xfailed": 1,
    "xpassed": 1,
    "total": 5
}

This seems to be the expected result, therefore I'll close the issue. Feel free to open a new issue if you find a bug with pytest-json-report.