warn when tests raise plain assertion errors without annotation

[RonnyPfannschmidt]

with the removal of reinterpretation, we start to expose hard to comprehend issues

for example in #1929 its not clear at first, that the import inside of the conftest causes the rewriting of the test module to fail

in order to protect users we need to make them aware of those subtle issues

edit additionally the documentation on importing code in conf-tests should be revised

[nicoddemus]

I agree it would be nice, but I have no idea how to do any kind of checking in this sense...

[RonnyPfannschmidt]

when collecting a test module we can check for attribute names starting with @pytest

[nicoddemus]

But test modules are always rewritten... The problem is other modules with asserts that we would like to rewrite too.

[RonnyPfannschmidt]

wrong - as far as i can tell from the errors in #1929 test modules do not get rewritten when importing them from a conftest first (so the rewritten conftest causes a non-rewritten import of the test module)

[nicoddemus]

Oh right... OK, now that makes sense, thanks.

On Thu, Sep 15, 2016 at 7:23 AM Ronny Pfannschmidt notifications@github.com wrote:

wrong - as far as i can tell from the errors in #1929 https://github.com/pytest-dev/pytest/issues/1929 test modules do not get rewritten when importing them from a conftest first (so the rewritten conftest causes a non-rewritten import of the test module)

— You are receiving this because you commented.

Reply to this email directly, view it on GitHub https://github.com/pytest-dev/pytest/issues/1930#issuecomment-247291592, or mute the thread https://github.com/notifications/unsubscribe-auth/ABCO_Hyekqm0SoB92Q8H2lms0yiWp7Gpks5qqRykgaJpZM4J6hM3 .

[obestwalter]

tested with pytest==3.6.5 on Python 3.6

For me this works as I would expect it without any of the workarounds mentioned in #1929 - so has this been fixed in some way or am I missing something?

code used (all modules in the same directory):

conftest.py:

from some_code import Foo

def pytest_assertrepr_compare(op, left, right):
    if isinstance(left, Foo) and isinstance(right, Foo) and op == "==":
        return ['Comparing Foo instances:',
                f'\tvalues: "{left.name}" != "{right.name}"']

some_code.py:

class Foo(object):
    def __init__(self, name):
        self.name = name

    def __eq__(self, other):
        return self.name == other.name

def do_something_with_foo():
    f1 = Foo("just some foo")
    f2 = Foo("another foo")
    return f1, f2

test_some_code.py:

from some_code import do_something_with_foo

def test_some_code():
    f1, f2 = do_something_with_foo()
    assert f1 == f2

ouput (as expected):

    def test_some_code():
        f1, f2 = do_something_with_foo()
>       assert f1 == f2
E       assert Comparing Foo instances:
E           values: "just some foo" != "another foo"

test_some_code.py:6: AssertionError

[RonnyPfannschmidt]

@obestwalter you never imported the test file in the conftest

i should add an xfailing test

[obestwalter]

oh ... the gist demonstrating the problem actually uses a test module called foo2.py containing the class under test in the same module ... my brain wasn't willing to accept that as reality, so it recalculated the code to what I think it should be like :)

For this (very exotic?) corner case, the problem still persists then.

[RonnyPfannschmidt]

@obestwalter its always a bit hilarious how our well meaning "auto-correct" brain mode automatically undoes issues users face that choose interesting or unexpected setups,