search

tarpas / pytest-testmon

Selects tests affected by changed files. Executes the right tests first. Continuous test runner when used with pytest-watch.
https://testmon.org
MIT License
800 stars 54 forks source link

Feature: Prioritise affected tests, but still run changed tests #197

Closed rgilton closed 1 year ago

rgilton commented 1 year ago

After making a change to my code, I'm interested in getting pytest to run the tests that are likely to fail first. Particularly useful when developing in a TDD way. I still want to run all tests so that I can be sure that everything definitely still works.

It'd be great to have an option that just prioritised running the tests that have been affected, instead of exclusively running the tests that have been affected. Similar behaviour to pytest's existing --ff flag, but with testmon's knowledge of what has changed.

At the moment, my workaround is to do this: pytest --testmon -x && pytest -x, but obviously this suffers from having to run through collection twice etc.

tarpas commented 1 year ago

--testmon-noselect -x should do just that! It doesn't deselect any tests but the sorting have been made the default behaviour a couple of releases ago. Please give it a try and report back if it seems buggy!

rgilton commented 1 year ago

Oh great! The help text for the --testmon-noselect flag doesn't make it sound like it'll do that though:

  --testmon-noselect    Run testmon but deactivate selection, so all tests
                        selected by other means will be collected and executed.
                        Forced if you use -k, -l, -lf, test_file.py::test_name

It ddoesn't mention reordering the tests at all -- might help if it did.

tarpas commented 1 year ago

It ddoesn't mention reordering the tests at all -- might help if it did.

Right, we'll fix that! Thanks.