Callable case generator with fixtures?

[lawschlosser]

I would like to be able to parameterize a test/function whose cases are generated by a class. The class would be able to use fixtures during the case-generation process (i.e. collection stage).

In the docs there is this example of a case generator (abbreviated):

from pytest_cases import parametrize, parametrize_with_cases

class CasesFoo:

    @parametrize(who=('you', 'there'))
    def case_simple_generator(self, who):
        return "hello %s" % who

@parametrize_with_cases("msg", cases=CasesFoo)
def test_foo(msg):
    assert isinstance(msg, str) and msg.startswith("hello")

This works, but I'd like:

1. the who values ('you', 'there') to come from a callable (rather than hard-coded in the decorator).
2. the callable would be able to use/rely-upon fixtures.

Below, I've added two superficial fixtures and a my_callable function to generate values for the who parameter.

import pytest
from pytest_cases import parametrize, parametrize_with_cases

@pytest.fixture()
def fixture1():
    pass

@pytest.fixture()
def fixture2():
    pass

def my_callable(fixture1, fixture2):
    return ['you', 'there']

class CasesFoo:

    @parametrize(who=my_callable)
    def case_simple_generator(self, who):
        return "hello %s" % who

@parametrize_with_cases("msg", cases=CasesFoo)
def test_foo(msg):
    assert isinstance(msg, str) and msg.startswith("hello")

This fails with:

_____________________ ERROR collecting test.py _____________________
test.py: in <module>
    class CasesFoo:
test.py: in CasesFoo
    @parametrize(who=my_callable)
/usr/local/lib/python2.7/dist-packages/pytest_cases/fixture_parametrize_plus.py:699: in parametrize
    hook=hook, debug=debug, **args)
/usr/local/lib/python2.7/dist-packages/pytest_cases/fixture_parametrize_plus.py:746: in _parametrize_plus
    argnames, argvalues = _get_argnames_argvalues(argnames, argvalues, **args)
/usr/local/lib/python2.7/dist-packages/pytest_cases/fixture_parametrize_plus.py:1119: in _get_argnames_argvalues
    kw_argnames, kw_argvalues = cart_product_pytest(tuple(args.keys()), tuple(args.values()))
/usr/local/lib/python2.7/dist-packages/pytest_cases/common_pytest.py:799: in cart_product_pytest
    argvalues_prod = _cart_product_pytest(argnames_lists, argvalues)
/usr/local/lib/python2.7/dist-packages/pytest_cases/common_pytest.py:818: in _cart_product_pytest
    for x in argvalues[0]:
E   TypeError: 'function' object is not iterable

Ok, so it looks like @parametrize doesn't want to be given a callable; It wants to be given a resolved list of values. No problem, I can just call my_callable, and pass the results to the @parametrize decorator, e.g.

class CasesFoo:

    @parametrize(who=my_callable())  # let's just call the callable
    def case_simple_generator(self, who):
        return "hello %s" % who

That results in this error:

test.py: in <module>
    class CasesFoo:
test.py: in CasesFoo
    @parametrize(who=my_callable())  # let's just call the callable
E   TypeError: my_callable() takes exactly 2 arguments (0 given)

...which I guess makes sense, but now I'm not sure how to proceed. I'm sure you've documented this (or a better/simpler approach) somewhere, so my apologies ahead of time. Any guidance/links would be much appreciated! Thank you!

Bonus puzzle

Continuing from that last attempt/failure, rather than augmenting the signature of my_callable (to indicate the need of fixture1 and fixture2), I figured maybe I could use @pytest.mark.usefixtures instead. And you know what, this actually "worked" !!

@pytest.mark.usefixtures("fixture1", "fixture2")
def my_callable():
    return ['you', 'there']

===================== test session starts ======================
platform linux2 -- Python 2.7.17, pytest-4.6.8, py-1.11.0, pluggy-0.13.1
rootdir: /tmp, inifile: /dev/null
plugins: cases-3.6.14, mock-2.0.0
collected 2 items

test.py ..             [100%]

=================== 2 passed in 0.04 seconds ===================

...except that it never actually ran/called the fixtures :frowning_face:

I would have thought there would be an error or warning if those fixtures were not run (since clearly I don't know what I'm doing), rather than silently failing (and in this case, passing)

Thanks so much for the work you've done!

[jgersti]

What you want to is not possible with pytest in general. Pytest test execution is split into several phases. First is the collection phase in which fixture and test functions are found., followed by the call phase where the fixtures and test are executed. Parametrization happens during the collection phase. This means all parameters must be known before any fixture is executed! There was a similar question at the pytest repo recently, https://github.com/pytest-dev/pytest/discussions/11359, maybe this can help you.

Regarding usefixture: This adds a mark on the function but pytest never sees this function in any context and so is not aware of the mark.

[lawschlosser]

@jgersti ah, bummer. that explains some things. Thanks for looking into this, I appreciate your help!

[smarie]

Indeed this is not possible with pytest. See also this https://github.com/smarie/python-pytest-cases/issues/235 Thanks @jgersti for providing the answer !