pytest-dev / pytest-rerunfailures

a pytest plugin that re-runs failed tests up to -n times to eliminate flakey failures
Other
378 stars 82 forks source link

Incompatible with pytest lazy_fixture plugin #109

Open MRigal opened 4 years ago

MRigal commented 4 years ago

Hi all,

Probably similarly to #99 , incompatible with pytest lazy_fixture plugin

In conftest.py, I have the following:

@pytest.fixture
def print_one:
    print('one')

@pytest.fixture
def print_two:
    print('two')

@pytest.fixture(
    params=[
        lazy_fixture('print_one'),
        lazy_fixture('print_two'),
    ]
)
def parametrize_prints():
    pass

And consider the following failing test:

def test_foo(parametrize_prints):
    assert False

Running pytest --reruns 1 -k test_foo -s prints out

oneRF
twoRF

Although the fixture are well identified by pytest in the prints test_file.py::test_foo[print_one]

The example here is trivial, but in my case, I want the system to run some tests at specific times and the tests always pass on re-run because lazy_fixture with freezegun is never executed. This makes the rerunfailures plugin actually unsuitable for us :-(

refeved commented 3 years ago

Can anyone have a look on this issue? I can also reproduce similar problem on my envrionment which causing rerun always fail.

sallner commented 3 years ago

@MRigal I have never used lazy_fixtures before. What would you expect in this case? That the fixture is generated a second time?

MRigal commented 3 years ago

@sallner yes for sure! That it re-runs the same fixture

MRigal commented 3 years ago

If you tell me where to dig, I could give it a look, but I have only little knowledge of the internals of pytest and the plugin environment