search

pytest-dev / pytest

The pytest framework makes it easy to write small tests, yet scales to support complex functional testing
https://pytest.org
MIT License
11.85k stars 2.64k forks source link

Emit a warning when using an unregistered mark via `-m` #10514

Open The-Compiler opened 1 year ago

The-Compiler commented 1 year ago

In #4826 (and earlier with --strict), we introduced warnings for applying unregistered marks, because otherwise something like:

@pytest.mark.sloww  # sic!
def test_slow():
    pass

would be a silent failure, with the test not being run with -m slow.

However, there is a similar issue when using marks: If you end up e.g. accidentally typing -m "not sloww", this won't actually skip the slow tests, but you might not actually notice that this mark doesn't exist. Similarly, if you do -m "mark1 or mark2_with_typo", it's tricky to recognize that less tests are run than what was actually intended.

IMHO, such usage should raise a warning (or even an error?).

The-Compiler commented 5 months ago

@poulami-sau, from #12166:

I am interested in working on this issue, but I'm having a hard time trying to simulate the conditions, and would appreciate some guidance on this!

Could you be more specific about what's giving you trouble? The easiest way to reproduce this would me -m doesnotexist, where pytest should warn that there is no registered marker with that name (just like when you add @pytest.mark.doesnotexisteither above a test). Currently, pytest only shows "no tests ran" instead.