pytest-dev / pytest-xdist

pytest plugin for distributed testing and loop-on-failures testing modes.
https://pytest-xdist.readthedocs.io
MIT License
1.46k stars 232 forks source link

last failed is not honored when collection error occurs #1087

Open kasium opened 4 months ago

kasium commented 4 months ago

Steps to reproduce

  1. create a folder names tests
  2. create a file test_foo.py with the content def test_1(): assert True
  3. create a file test_bar.py with the content
    import pytest
    @pytest.skip("reason")
    def test_2():
    assert False
  4. Execute pytest tests/ -n2 -vv
  5. Execute pytest tests/ -n2 --lf -vv --last-failed-no-failures none

As you can see in the logs test_bar cannot be collected due to a collection error. However in the second run also test_foo is exeuted again which is not expected. When pytest is executed w/o pytest-xdist the execution fails directly bc of the collection error. If the collection error is removed (still assert False will lead to an error), the behavior is as expected.

I use the latest pytest, pytest-xdist version and python 3.12.1