Using fixtures in pytest.mark.parametrize

[pytestbot]

Originally reported by: Florian Rathgeber (BitBucket: frathgeber, GitHub: frathgeber)

---

I often have a use case like the following contrived example:

@pytest.fixture
def a():
    return 'a'

@pytest.fixture
def b():
    return 'b'

@pytest.mark.parametrize('arg', [a, b])
def test_foo(arg):
    assert len(arg) == 1

This doesn't currently do what's intended i.e. arg inside the test function is not the fixture value but the fixture function.

I can work around it by introducing a "meta fixture", but that's rather ugly:

@pytest.fixture(params=['a', 'b'])
def arg(request, a, b):
    return {'a': a, 'b': b}[request.param]

def test_foo(arg):
    assert len(arg) == 1

It would be convenient if a syntax like in the first case was supported.

---

• Bitbucket: https://bitbucket.org/pytest-dev/pytest/issue/349

[pytestbot]

Original comment by Matthias Geier (BitBucket: geier, GitHub: geier):

---

This would be a great feature!

I found another (I don't know if more or less ugly) work-around:

#!python

@pytest.mark.parametrize('arg', ['a', 'b'])
def test_foo(arg, request):
    val = request.getfuncargvalue(arg)
    assert len(val) == 1

This doesn't work, however, with parametrized fixtures.

BTW, it would also be great if fixtures were supported in the params argument of pytest.fixture.

[pytestbot]

Original comment by Floris Bruynooghe (BitBucket: flub, GitHub: flub):

---

Tentatively assigning this to me as I think I might be able to come up with a reasonable patch. It'll probably take me a long while though so don't let that discourage anyone else from working on this, assigning it more as a way of not forgetting about it.

[pytestbot]

Original comment by Praveen Shirali (BitBucket: praveenshirali, GitHub: praveenshirali):

---

The quoted examples work because functions a and b are part of the same module as test_foo, and within the scope of the example, the parametrization should work even if @pytest.fixture decorator isn't present around functions a and b. They are getting used as regular python functions and not as pytest fixtures. Note that fixtures can also be defined in external modules like conftest.py.

Another alternative to the above example is to directly call these functions in the list.

#!python

@pytest.mark.parametrize('arg', [a(), b()])
def test_foo(arg):
    assert len(arg) == 1

[pytestbot]

Original comment by BitBucket: dpwrussell, GitHub: dpwrussell:

---

This would be an awesome feature.

@praveenshirali I don't think your alternative is used as a fixture, it just calls the fixture function. So it would be run repeatedly. You would also have to specify the arguments to the fixture if there were any which could begin the cycle over again if they are also fixtures.

[mgeier]

Here's a related stackoverflow question: http://stackoverflow.com/questions/24340681/how-to-concatenate-several-parametrized-fixtures-into-a-new-fixture-in-py-test

[jlmenut]

Yes, I would like very much to have this feature also. Maybe a line in the doc explaining it's not possible for the moment would be useful also.

[kevincox]

It would also be killer if this supported parameterized fixtures generating the product of the fixtures. Although this might be a little much.

@pytest.fixture(params=["1", " ", 1, True, [None], {1:2}])
def truthy(request):
    return request.param

@pytest.fixture(params=[False, None, "", 0, [], {}])
def falsey(request):
    return request.param

@pytest.mark.parameterize("val,res", [
    (truthy, True),
    (falsey, False),
])
def test_bool(val, res)
    assert bool(val) is res

[SUNx2YCH]

+1 for this feature. BTW, combining Florian Rathgeber and Matthias Geier solutions we can get a bit nicer "meta fixture":

@pytest.fixture
def a():
    return 'a'

@pytest.fixture
def b():
    return 'b'

@pytest.fixture(params=['a', 'b'])
def arg(request):
    return request.getfuncargvalue(request.param)

def test_foo(arg):
    assert len(arg) == 1

[rabbbit]

+1 on this.

I'm currently writing tests that look like:

@pytest.fixture
def my_fixture():
      return 'something'

@pytest.fixture
def another_fixture(my_fixture):
      return {'my_key': my_fixture}

def yet_another_fixture():
     return {'my_key': None}

@pytest.mark.parametrize('arg1, arg2', [
    (5, another_fixture(my_fixture())),
    (5, yet_another_fixture()),
)
def my_test(arg1, arg2):
    assert function_under_test(arg2) == arg1

and that's rather ugly.

[RonnyPfannschmidt]

@rabbbit your example is structurally wrong and runs fixture code at test importation time

[rabbbit]

@RonnyPfannschmidt I know - and that's why I'd like to be able to use fixtures in parametrize? And that would be awesome.

My example is wrong, but it follows the guideline of "always use fixtures". Otherwise we'd end up with fixtures in normal tests, and workarounds in parametrized tests.

Or is there a way of achieving this already, outside of dropping parametrize and doing 'if/elses' in the test function?

[RonnyPfannschmidt]

There is a upcoming proposal wrt "merged" fixtures, there is no implementation yet

[nicoddemus]

For reference: #1660

[rabbbit]

ok, I don't understand python.

If the below works:

@pytest.fixture
def my_fixture
    return 1

def test_me(my_fixture):
    assert 1 == my_fixture

wouldn't the below be simpler? And an exact equivalent?

@pytest.fixture
def my_fixture
    return 1

@pytest.mark.parametrize('fixture', [my_fixture])
def test_me(fixture):
    assert 1 == my_fixture

Am I wrong to think that mark.parametrize could figure out whether an argument is a pytest.fixture or not?

[RonnyPfannschmidt]

atm parametrize cannot figure it, and it shouldnt figure it there will be a new object to declare a parameter will request a fixture/fixture with parameters

some documentation for that is in the features branch

[rabbbit]

yeah, I read the proposal.

I'm just surprised you're going with pytest.fixture_request(' default_context')it feels very verbose?

after all,

@pytest.fixture
def my_fixture
    return 1

def test_me(my_fixture):
    assert 1 == my_fixture

could also turn to

@pytest.fixture
def my_fixture
    return 1

def test_me(pytest.fixture_request(my_fixture)):
    assert 1 == my_fixture

but that's not the plan, right?

On 5 July 2016 at 16:17, Ronny Pfannschmidt notifications@github.com wrote:

atm parametrize cannot figure it, and it shouldnt figure it there will be a new object to declare a parameter will request a fixture/fixture with parameters

some documentation for that is in the features branch

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/pytest-dev/pytest/issues/349#issuecomment-230490966, or mute the thread https://github.com/notifications/unsubscribe/AARt0kUxVSL1GMeCtU-Kzy3Pg80mW7Ouks5qSmdmgaJpZM4FEMDj .

[RonnyPfannschmidt]

thats not even valid python syntax

the fixture-request would be used as a parameter value to tell py.test "use the value of a fixture you create later on"

there are already parametzw using examples, and its not overly verbose

[kibernick]

It would be really convenient to have this functionality, perhaps along the lines of "LazyFixture" in pytest-factoryboy

[RonnyPfannschmidt]

@kibernick we already did put a outline of possible implementations into the documentation

we just need time or a person implementing it

[Brachi]

@RonnyPfannschmidt can you link to that part of the documentation you mention? Can't find it. Edit: nevermind. http://doc.pytest.org/en/latest/proposals/parametrize_with_fixtures.html

[TvoroG]

@RonnyPfannschmidt, can you please check this out https://github.com/TvoroG/pytest-fixture-mark? Need some feedback

[Brachi]

@TvoroG good work. Currently this seems to be failing. Is it possible to support it as well?

import pytest                                                                  

@pytest.fixture(params=[1, 2, 3])                                              
def one(request):                                                              
    return str(request.param)                                                  

@pytest.fixture                                                                
def two():                                                                     
    return 4                                                                   

@pytest.fixture(params=[                                                       
    pytest.mark.fixture('one'),                                                
    pytest.mark.fixture('two')                                                 
])                                                                             
def some(request):                                                             
    return request.param                                                       

def test_func(some):                                                           
    assert some in {'1', '2', '3', 4}

[TvoroG]

@Brachi, thanks for catching it! It works now, but more nested structures need some dependency sorting to instantiate fixtures in correct order. I'll update the plugin code when i'm done.

[TvoroG]

@Brachi, I fixed it. Let me know if there is more such cases when plugin is failing

[Brachi]

@TvoroG great, thanks for the quick reply. I tested it a little more and here's another contrived example that doesn't work, based on a real use case (where one actually returns an object)

import pytest

@pytest.fixture(params=[1, 2, 3])
def one(request):
    return request.param

@pytest.fixture(params=[pytest.mark.fixture('one')])
def as_str(request):
    return str(request.getfixturevalue('one'))

@pytest.fixture(params=[pytest.mark.fixture('one')])
def as_hex(request):
    return hex(request.getfixturevalue('one'))

def test_as_str(as_str):
    assert as_str in {'1', '2', '3'}

def test_as_hex(as_hex):
    assert as_hex in {'0x1', '0x2', '0x3'}

# fails at setup time, with ValueError: duplicate 'one'
def test_as_hex_vs_as_str(as_str, as_hex):
    assert int(as_hex, 16) == int(as_str)

[nicoddemus]

@TvoroG, pytest-fixture-mark seems very nice! I hope I get the chance to try it soon! 😄

Perhaps discussion specific to it should be moved to https://github.com/TvoroG/pytest-fixture-mark thought? 😉

[rabbbit]

Hoping that this could get merged into pytest would be too much, right? :)

On 4 Oct 2016 23:24, "Bruno Oliveira" notifications@github.com wrote:

@TvoroG https://github.com/TvoroG, pytest-fixture-mark seems very nice! I hope I get the chance to try it soon! 😄

Perhaps discussion specific to it should be moved to https://github.com/TvoroG/pytest-fixture-mark thought? 😉

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/pytest-dev/pytest/issues/349#issuecomment-251518117, or mute the thread https://github.com/notifications/unsubscribe-auth/AARt0vvrzeWro3wD9-G1nKn1H89OrLCJks5qwsP8gaJpZM4FEMDj .

[nicoddemus]

I like how it played out, but @hackebrot and @hpk42 had something else in mind during the sprint. 😁 Perhaps they can comment on this.

[Brachi]

Perhaps discussion specific to it should be moved to

I was thinking the same thing before posting, but probably the use cases and limitations are useful as well for other implementations.

[RonnyPfannschmidt]

i think its critical to make the fixture request NOT be a mark object as well

[frol]

@TvoroG I have tried your extension and it works great! Thank you!

[brianbruggeman]

I have a less contrived example....

@pytest.fixture
def today():
    import datetime as dt
    today = dt.datetime.utcnow()
    return today.strftime('%Y%m%d')

@pytest.fixture
def yesterday():
    import datetime as dt
    today = dt.datetime.utcnow()
    yesterday = today - dt.timedelta(days=1)
    return yesterday.strftime('%Y%m%d')

And I kind of expected/wanted to use them this way:

@pytest.mark.usefixtures("today", "yesterday")
@pytest.mark.parametrize("date, expected", [
    (None, False),
    (today, False),
    (yesterday, True),
])
def test_my_date_function(date, expected):
    from my_package import date_function

    assert date_function(date) == expected

In the example above, mypackage.date_function actually keys off of the current date to determine the return value. I have a few of these functions I'd like to test like the above.

[The-Compiler]

@brianbruggeman for that example, you could make them normal functions and simply call them in @pytest.mark.parametrize.

[lopter]

What about a similar example where the assertion would have to be is instead of ==?

On January 5, 2017 6:50:39 AM GMT+01:00, Florian Bruhin notifications@github.com wrote:

@brianbruggeman for that example, you could make them normal functions and simply call them in @pytest.mark.parametrize.

-- You are receiving this because you are subscribed to this thread. Reply to this email directly or view it on GitHub: https://github.com/pytest-dev/pytest/issues/349#issuecomment-270569273

-- Louis Opter

[RonnyPfannschmidt]

we already have a some documented future plans at http://doc.pytest.org/en/latest/proposals/parametrize_with_fixtures.html

[Alex-Bogdanov]

Sorry, when this 2 features will be released?http://doc.pytest.org/en/latest/proposals/parametrize_with_fixtures.html pytest.define_combined_fixture() pytest.fixture_request()

[The-Compiler]

@Alex-Bogdanov it's nothing more than an idea/proposal currently.

[Alex-Bogdanov]

@The-Compiler any ideas how to push forward it? cause it would be very helpful feature

[nicoddemus]

@Alex-Bogdanov the best way would be to start a PR yourself; AFAIK all maintainers are working in other features/bug fixes for 3.1.

[Alex-Bogdanov]

@nicoddemus Got you. Starting

[binary10]

I had to find a workaround for this issue in order to share a resource in my module. The idea is to dynamically generate a list from which test cases are then generated. The following things didn't work:

• Using pytest fixtures and providing the fixture name to pytest.mark.parametrize (original issue description)
• Calling the fixture function in pytest.mark.parametrize (won't work for fixtures that call other fixtures)
• Setting up a variable in setup_module, then calling it in pytest.mark.parametrize (it appears the decorator is called before the module is initialized)
• Using conftest.py and metafunc (too convoluted and meta)

What worked: The simplest approach I found is to forget using pytest fixtures at all and to create a dumb "fixture". I simply created a "constant" variable (global to the module) for my resource. I can now pass it to the decorator without issues. I lose the smart aspects of pytest fixtures (like collecting tests that relate to that fixture), but getting the test output I need is more important than this.

To only support fixtures in the test body seems like a big oversight. It should be possible to use fixtures anywhere. Wouldn't fixtures need to be initialized before the tests are collected anyway? If this isn't the case then why?

[phobson]

Here's how I worked around this:

class Thing:
    def __init__(self, datadir, errorfile):
        self.datadir = datadir
        self.errorfile = errorfile

@pytest.fixture(params=['test1.log', 'test2.log'])
def thing(request):
    with tempfile.TemporaryDirectory() as datadir:
        errorfile = os.path.join(datadir, request.param)
        yield Thing(datadir=datadir, errorfile=errorfile)

def test_thing_datadir(thing):
    assert os.path.exists(thing.datadir)

[Sup3rGeo]

Related to this may be my question on stackoverflow.

Basically it would be nice to parametrize depending on a fixture like this:

datasetA = [data1_a, data2_a, data3_a]
datasetB = [data1_b, data2_b, data3_b]

@pytest.fixture(autouse=True, scope="module", params=[datasetA, datasetB])
def setup(dataset):
    #do setup (only once per dataset)
    yield
    #finalize

#dataset should be the same instantiated for the setup
@pytest.mark.parametrize('data', [data for data in dataset]) 
def test_data(data):
    #do test

It doesn't seem to be possible to do this as of today and I think it is related to parametrizing on fixtures too.

[RonnyPfannschmidt]

@Sup3rGeo if the dataset size is always the same you can do something like have a independent parameter thats just the index of the element

you can even account for different sizes by going for the max size and skipping the missing elements if the cost is manageable

[offbyone]

I'm looking at something that I ... think is related to this? What I want to do is this:

@pytest.fixture(scope='session')
def bootstrap():
    # do something super slow

@pytest.fixture
def config_dir(bootstrap, tmpdir):
    # do something faster with the bootstrap data

def pytest_generate_tests(metafunc):
    # generate some tests based off the current value of the `config_dir` fixture.

I don't see anything in here that is quite what I'm looking for, but I'm open to hearing otherwise.

[ndevenish]

My use case is: I want to parametrize on every file in a regression data folder. Except this folder can move, so the location is (usually) passed to tests as a fixture. Contrived example of what I'm currently doing to work around this:

@pytest.fixture
def regression_data():
  return "some_folder_location"

# Generate list of all files in the regression folder
all_files = [os.path.join(regression_data(), x) for x in os.listdir(regression_data())]

# Convert each parameter value into a fixture
@pytest.fixture(params=all_files)
def regression_image(request):
  return request.param

def test_every_file(regression_image):
  # Test the single image regression_image

e.g. I have to either duplicate code or from ..conftest import regression_data or similar to share the logic for location.

What i feel would be ideal is if I could pass through the parametrization in the fixture (you can send it in, but not out), where other fixtures are available. Perhaps e.g. something like

@pytest.fixture
def regression_image(regression_data):
  all_files = [] #Generate list of files based on regression_data
  return pytest.parametrize(params=all_files) 

or even, to borrow the initial example:

@pytest.fixture
def a():
  pass

@pytest.fixture
def b():
  pass

@pytest.fixture
def regression_image(request, a, b):
  return pytest.parametrize(params=[a, b]) 

or maybe even aggregating other parametrizations?

@pytest.fixture(params=["fileA", "fileB"])
def example_file(location, param):
  return pytest.parametrize(params=[os.path.join(location, param)])

I know people seem to be leaning towards the (with implementation) lazy-fixtures approach, but would this even be possible, or is there e.g. some ordering problem where parameters are resolved before fixtures, or something similar.

[nicoddemus]

Hi @ndevenish,

Your examples are look nice and well written, thank you.

Under the current design, pytest needs to generate all items during the collection stage. Your examples generate new items during the session testing stage (during fixture instantiation), which is not currently possible.

[nluetzge]

Hi! Is there any development in this issue? I would really love to be able to use fixtures in the parametrization. At the moment it is not possible and the workarounds are not optimal.

Cheers, Nora

[goodusermf]

I have a not so pretty work around that works for me; hope someone finds it useful.

class MyTests():
     @staticmethod
     def get_data()
           items_list = ["a", "b", "c", "d"]
            return items_list

      @pytest.mark.parametrize('letter', get_data.__func__())
      def test_my_test(letter)
           # consume letter in your test

[dtomas]

Possible workaround, involving indirect parametrization and passing pytest.mark.usefixtures() to pytest.param via its marks argument:

import pytest

@pytest.fixture
def a():
    return 'a'

@pytest.fixture
def b():
    return 'b'

@pytest.fixture
def arg(request):
    return request.getfixturevalue(request.param)

@pytest.mark.parametrize('arg', [
    pytest.param('a', marks=pytest.mark.usefixtures('a')),
    pytest.param('b', marks=pytest.mark.usefixtures('b')),
], indirect=True)
def test_foo(arg):
    assert len(arg) == 1

Tested with pytest 3.2.5 and 3.8.0. Can some pytest dev confirm that this is supposed to work (and continue to work), or are there some issues one could run into further down that road?