pytest_generate_tests doesn't play with pytest.mark.parametrize

[philpep]

Consider this test case:

import pytest

CALL_COUNT = 0 

@pytest.fixture(scope="module")
def fixture(_dynamic_param):
    global CALL_COUNT
    CALL_COUNT += 1 

def pytest_generate_tests(metafunc):
    if "_dynamic_param" in metafunc.fixturenames:
        metafunc.parametrize("_dynamic_param", ["foo"], scope="module")

@pytest.mark.parametrize("param", ["bar", "zaz"])
def test_1(fixture, param):
    global CALL_COUNT
    assert CALL_COUNT == 1

The output is:

% py.test test_bug.py
platform linux2 -- Python 2.7.9 -- py-1.4.30 -- pytest-2.7.2
rootdir: /tmp, inifile: 
collected 2 items 

test_bug.py .F

== FAILURES ==
__ test_1[foo-zaz] __

fixture = None, param = 'zaz'

    @pytest.mark.parametrize("param", ["bar", "zaz"])
    def test_1(fixture, param):
        global CALL_COUNT
>       assert CALL_COUNT == 1
E       assert 2 == 1

The fixture is called twice here, howerver it's a module scoped fixture so I expect only one call.

The bug doesn't occur when writting two tests instead of using pytest.mark.parametrize or when using @pytest.fixture(scope="module", param=["foo"] instead of pytest_generate_tests.

Maybe related to #635

Thanks

[RonnyPfannschmidt]

@philpep does this also happen when using indirect=True?

[acloyd]

I've run into the same issue. I've found that it doesn't have to use both pytest_generate_tests and pytest.mark.parametrize. The issue comes about with any fixture that uses a parametrized parameter, along with another parametrized parameter being used in the same test function. The following reproduces the above example but with using only pytest_generate_tests:

import pytest

CALL_COUNT = 0 

def pytest_generate_tests(metafunc):
    if "fixture_param" in metafunc.fixturenames:
        metafunc.parametrize("fixture_param", ["foo"], scope="module")
    if "test_param" in metafunc.fixturenames:
        metafunc.parametrize("test_param", ["bar", "zaz"], scope="module")

@pytest.fixture(scope="module")
def fixture(fixture_param):
    global CALL_COUNT
    CALL_COUNT += 1

def test_1(fixture, test_param):
    global CALL_COUNT
    assert CALL_COUNT == 1

[tomioueda]

I'm running into this issue as well, is there a workaround for this?

[philpep]

@RonnyPfannschmidt indeed that doesn't occur with indirect=True

This test pass:

import pytest

CALL_COUNT = 0 

@pytest.fixture(scope="module")
def fixture(request):
    global CALL_COUNT
    CALL_COUNT += 1 

def pytest_generate_tests(metafunc):
    if "fixture" in metafunc.fixturenames:
        metafunc.parametrize("fixture", ["foo"], indirect=True, scope="module")

@pytest.mark.parametrize("param", ["bar", "zaz"])
def test_1(fixture, param):
    global CALL_COUNT
    assert CALL_COUNT == 1

[Stranger6667]

Unfortunately #1766 didn't fix the issues here :(

[RonnyPfannschmidt]

import pytest
import os

HAS_CAR = bool(os.environ.get("HAS_CAR"))

def pytest_generate_tests(metafunc):
    if 'car' in metafunc.fixturenames:
        metafunc.parametrize(
            'car', ['red', 'green'],
            scope='module', indirect=HAS_CAR)

if HAS_CAR:
    @pytest.fixture(scope='module')
    def car(request):
        return request.param

@pytest.yield_fixture(scope='module')
def prepared_car(car):
    print("\n{} CAR PREPARED".format(car))
    yield car
    print("\n{} CAR TAKEN AWAY".format(car))

@pytest.mark.parametrize('wheel', [1, 2, 3], scope='function')
def test_car_wheels(prepared_car, wheel):
    print("CHECKING {} CAR, WHEEL NUMBER {}".format(prepared_car, wheel))

def test_car_steering(prepared_car):
    print("CHECKING {} CAR, STEERING WHEEL".format(prepared_car))

is a more extended example

[jpsnyder]

It looks like I am running into this issue too, which is causing my heavy setup/teardown to be run on each function instead of per session. Is there any workaround for this?