tarpas / pytest-testmon

Selects tests affected by changed files. Executes the right tests first. Continuous test runner when used with pytest-watch.
https://testmon.org
MIT License
800 stars 54 forks source link

Changes to global / class variables are ignored (if no method of their module is executed) #191

Open janbernloehr opened 1 year ago

janbernloehr commented 1 year ago

Suppose you have the following files

# mymodule.py
foo_bar = "value"

class MyClass:
    FOO = "bar"

# test_mymodule.py
from mymodule import MyClass, foo_bar

def test_module():
    assert foo_bar == "value"
    assert MyClass.FOO == "bar"

Now running pytest --testmon test_mymodule.py does not rerun test_module() when the value of MyClass.FOO or foo_bar is changed. Even worse, FOO can be completely removed, e.g.

class MyClass:
    pass

without triggering a re-run.

When running pytest --cov manually, there seems to be a trace

image

tarpas commented 1 year ago

Unfortunately this is quite a fundamental limitation. It's not impossible to fix but hard.

tarpas commented 1 year ago

It's also worth mentioning that if there is at least one function or method from mymodule used in test_mymodule then any change in mymodule is taken into account.

janbernloehr commented 1 year ago

This issue is unfortunately still present with testmon 2.0.9