wntrblm / nox

Flexible test automation for Python
https://nox.thea.codes
Apache License 2.0
1.31k stars 151 forks source link

feat: allow setting tags on parametrized sessions #832

Closed living180 closed 3 weeks ago

living180 commented 4 months ago

To allow more fine-grained session selection, allow tags to be set on individual parametrized sessions via either a tags argument to the @nox.parametrize() decorator, or a tags argument to nox.param() (similar to how parametrized session IDs can be specified). Any tags specified this way will be added to any tags passed to the @nox.session() decorator.

A couple of examples of how this could be used:

@nox.session
@nox.parametrize(
    "db_engine", [
        nox.param("sqlite", tags=["sqlite"])
        nox.param("mysql", tags=["mysql"])
        nox.param("postgresql", tags=["psql"])
    ]
)
@nox.parametrize(
    "django", [
        nox.param(">=3,<4", tags=["django3"])
        nox.param(">=4,<5", tags=["django4"])
        nox.param(">=5,<6", tags=["django5"])
    ]
)
def test(session, db_engine, django):
    pass

In this case, a developer could run nox -t sqlite to run just the tests with all versions of Django but only using the SQLite backend.

Here's a more complex example:

def generate_params():
    for python in ["3.10", "3.11", 3.12"]:
        for django in [3, 4, 5]:
            tags = []
            if python == "3.12" and django == 5:
                tags.append("quick")
            if python == "3.12" or django == 5:
                tags.append("standard")
            yield nox.param([python, django], tags)

@nox.session
@nox.parametrize(
    ["python", "django"], generate_params(),
)
def test(session, django):
    pass

This tags the Python 3.12/Django 5 combination with the quick tag, allowing the developer to run a quick sanity check on the code using a single entry from the test matrix. It also tags any combination of Python 3.12 or Django 5 with the standard tag, allowing the developer to run the tests using Python 3.12 with all versions of Django along with all versions of Python with Django 5, which should give fairly comprehensive test coverage while only having to run five entries from the test matrix instead of nine.

living180 commented 4 months ago

If this feature/approach is acceptable, I will update the PR to add the necessary documentation updates as well.

henryiii commented 1 month ago

Seems reasonable.