JetBrains / teamcity-messages

Python Unit Test Reporting to TeamCity
https://pypi.python.org/pypi/teamcity-messages
Apache License 2.0
138 stars 81 forks source link

add support for pytest-subtests #281

Open radoering opened 5 months ago

radoering commented 5 months ago

Objective

Select pytest as test runner in PyCharm, even if unit tests that use the subTest feature are executed and still get separate items for the subtests in the report.

Background

unittest can only execute unittest tests but not pytest tests. pytest can execute unittest tests as well as pytest tests. However, the (unittest specific) subTest feature is not supported, so that the complete tests are aborted as soon as the first subtest fails. There is a pytest plugin pytest-subtests to support subtests. With the plugin, subtests are supported when executing pytest in a shell. However, PyCharm still does not support subtests if pytest is chosen as test runner because the teamcity pytest plugin does not support this feature.

Results

Example script tests/test_it.py:

class MyTest(unittest.TestCase):
    def test_this(self):
        l = ["a", "b", "c"]
        for x in l:
            with self.subTest(f"test {x}"):
                assert x != "b"

Execute the example tests/test_it.py...

... with test runner unittest:

unittest

-> separate sub tests are displayed

... with test runner pytest (without pytest-subtests):

pytest without pytest-subtests

-> no sub tests are displayed

... with test runner pytest (with pytest-subtests, unpatched teamcity):

pytest with pytest-subtests and unpatched teamcity

-> broken end state looks like tests never finish

... with test runner pytest (with pytest-subtests, patched teamcity):

pytest with pytest-subtests and patched teamcity

-> same as with test runner unittest 🤩