pytest-dev / pytest

The pytest framework makes it easy to write small tests, yet scales to support complex functional testing
https://pytest.org
MIT License
11.96k stars 2.66k forks source link

`--fixtures-per-test`: Exclude parametrizing pseudo fixtures? #11295

Open The-Compiler opened 1 year ago

The-Compiler commented 1 year ago

With something like:

import pytest

def test_one(monkeypatch):
    pass

@pytest.mark.parametrize("x", [1, 2])
def test_two(x):
    pass

running pytest --fixtures-per-test results in:

-------------------------- fixtures used by test_one ---------------------------
----------------------------------- (x.py:4) -----------------------------------
monkeypatch -- .../_pytest/monkeypatch.py:30
    A convenient fixture for monkey-patching.

------------------------- fixtures used by test_two[1] -------------------------
----------------------------------- (x.py:7) -----------------------------------
x -- .../_pytest/fixtures.py:352
    no docstring available

------------------------- fixtures used by test_two[2] -------------------------
----------------------------------- (x.py:7) -----------------------------------
x -- .../_pytest/fixtures.py:352
    no docstring available

IMHO, the output for x should be excluded here - while it is technically a (pseudo) fixture internally, showing it like that is rather confusing for users. It doesn't make sense to show it as a fixture without a docstring which is "defined" here:

https://github.com/pytest-dev/pytest/blob/d9bf9dbec104af1e3a2c02b04f4e1d501d5d86a2/src/_pytest/fixtures.py#L351-L352

WarrenTheRabbit commented 1 year ago

You think that

-------------------------- fixtures used by test_one ---------------------------
----------------------------------- (x.py:4) -----------------------------------
monkeypatch -- .../_pytest/monkeypatch.py:30
    A convenient fixture for monkey-patching.

is clearer to the user?

I am a new user (and a new developer) and I certainly find that clearer.

The expectation I have for a fixture is that

  1. its author has the option of including a docstring that explains its purpose
  2. fixtures written by the pytest devs always have docstrings
  3. fixtures are created using the pytest.fixture() decorator

If I see a fixture without a docstring, I'd assume I had created it somewhere. Which I would have. But not with the pytest.fixture() decorator.

For me, it would be clearer if either

  1. the output for x is removed from --fixtures-per-test's output; or
  2. the output for x is kept but a docstring tells me it has been generated by pytest to pass an element from the pytest.mark.paramaterize()'s parameter list to the test function
The-Compiler commented 1 year ago

Yeah, IMHO those pseudo-fixtures just shouldn't show up there at all.

WarrenTheRabbit commented 1 year ago

I'm eager to start contributing. But am still a novice programmer. The question of my capability aside, I'm reading through the contribution guide right now to understand what processes of governance/discussion/consensus/etc must turn over their gears before a change of that kind could be officially proposed/developed on.

The-Compiler commented 1 year ago

For a smaller change like this, an issue like we have here is a good place for this discussion to take place. Given that there hasn't been any push-back, I'd suggest you go ahead with a pull request! I'm not too familiar with the fixture code myself, but based on gut feeling, this sounds like a good first issue to tackle. Happy to help if you get stock somewhere!

WarrenTheRabbit commented 1 year ago

Amazing. =) I have things on for the next few days but when I get the chance I'll go through the steps in the Long version of Preparing Pull Requests. Then, once I've implemented a change, I'll do step 11 (submit the pull request through GitHub) with something like:

head-fork: WarrenTheRabbit/pytest
compare: exclude-parameterising-fixtures-from-fixtures-per-test-output

base-fork: pytest-dev/pytest
base: main
WarrenTheRabbit commented 1 year ago

It has been over a week since I last commented on this issue so I thought I would share an update as I've made no Pull Request either:

I first mention the idea of doing a case study in I want to encourage beginners to contribute by making refactoring pull requests:

I think outreach and documentation of all stripes could achieve this. For example, one documentation idea is Case Study documents. Case Study documents would give an example of how a beginner (me) learned how to contribute to pytest. An example of that could be: documenting the work I do on https://github.com/pytest-dev/pytest/issues/11295.