search

JetBrains / teamcity-messages

Python Unit Test Reporting to TeamCity
https://pypi.python.org/pypi/teamcity-messages
Apache License 2.0
137 stars 81 forks source link

pytest: fix assert diffs #247

Closed ikonst closed 3 years ago

ikonst commented 3 years ago

In #210 I've tried to capture comparisons in pytest assertions in order to propagate them as diffs. The change in #210 attempts to reverse the formatting changes done by pytest, but it's not reliable and still leads to wrong behavior for multi-line changes.

A simpler and more reliable approach is implemented here. We stash the assertions within the assertion handler, and then retrieve them in pytest_runtest_logreport. This fixes handling of multiline comparisons. I've been using this version in my PyCharm development for months now, and it's been providing me flawless diffs in all kinds of complex cases.

P.S. The introduction of the "global" variable current_test_item is a little messy but pytest doesn't provide a nicer way -- see discussion in https://github.com/pytest-dev/pytest/issues/8740. If you look at the underlying pytest implementation (util._reprcompare), you'd see it's effectively assuming one test at a time.

ikonst commented 3 years ago

@mikekidya can take a look?

throwable-one commented 3 years ago

Hi @ikonst . Thank you for this change. Could you please add test that reproduces this problem, so we can be sure not to break it by some change in future?

ikonst commented 3 years ago

I'll try to add one. (I think I tried before and ran into some issues running the test suite.)

ikonst commented 3 years ago

I've added a regression test which you can see failing here.

I've reverted the fix and now it should pass again.

mikekidya commented 3 years ago

@ikonst Thank you for you contribution!

throwable-one commented 3 years ago

Also fixes https://github.com/JetBrains/teamcity-messages/issues/173