pytest-dev / pytest

The pytest framework makes it easy to write small tests, yet scales to support complex functional testing
https://pytest.org
MIT License
11.96k stars 2.66k forks source link

`params` on Fixture are not always detected during test generation #11350

Open jgersti opened 1 year ago

jgersti commented 1 year ago

Issue

params on fixtures that are used by a shadowed but used other fixture are not always detected during test generation and subsequent test execution fails with an error.

Example

@pytest.fixture(params=(1,))
def b(): pass

@pytest.fixture(name="a")
def a1(b):  pass

@pytest.fixture(name="a")
def a2(a): pass

def test_deep_params(a): pass
> pytest -q -vvv
============================= test session starts =============================
platform win32 -- Python 3.11.2, pytest-7.4.0, pluggy-1.2.0 -- xxx
cachedir: .pytest_cache
rootdir: xxx
collecting ... 
collected 1 item

::test_deep_params ERROR

=================================== ERRORS ====================================
_____________________ ERROR at setup of test_deep_params ______________________
The requested fixture has no parameter defined for test:
    ::test_deep_params

Requested fixture 'c' defined in:
:24

Requested here:
.venv\Lib\site-packages\_pytest\fixtures.py:712
=========================== short test summary info ===========================
ERROR test_base.py::test_deep_params - Failed: The requested fixture has no p...
============================== 1 error in 0.07s ===============================

Cause

During calculation of the fixture closue only the the arguments of the last FixtureDef for a fiven argument are expanded. The sequences of FixtureDefs are ordered alphabetical by their name (qualname?) of their function object (e.g. swapping the names of a1and a2 works). In the example above this result in the arguments of a1 being ignored and b never being picked up on, resulting in the error at execution time.

jgersti commented 1 year ago

This is in essence a more boiled down version of #2844. Calculation of the fixture closure (and following that parametrization) are to eager.

jgersti commented 1 year ago

In #5303 @bluetech identified the same base cause (closure calculation) resulting in a wrong fixture initialisation order instead of an error.

TanyaAgarwal28 commented 1 year ago

Hi @Zac-HD , I would like to pick this up. Will share the solution in a while.

TanyaAgarwal28 commented 1 year ago

As far as I can see here the issue is multiple fixtures with the same name (a in this case) but different parameters. The last fixture definition with the same name takes precedence, and its arguments override the earlier fixture's arguments. So Is this not a issue of user like user should enter the unique fixture names

TanyaAgarwal28 commented 1 year ago

If we want to handle this in a code then we can append a unique identifier to each fixture name based on its parameterization. This will ensure that fixtures with different parameterizations are treated as separate fixtures. @Zac-HD @RonnyPfannschmidt

RonnyPfannschmidt commented 1 year ago

A key problem here is that fixtures may be overridden by fixtures of the same name, and different fixture of that override chain may have different parameterization

Currently Parameterization just goes by name and falls flat on both overrides

We are quite far away from parameterization by fixture definition/alias