RonnyPfannschmidt
commented
6 years ago i suspect the keyword expression is matching the parametrize mark as well
Open rasschaert opened 6 years ago
RonnyPfannschmidt
commented
6 years ago i suspect the keyword expression is matching the parametrize mark as well
rasschaert
commented
6 years ago Hah! Good catch!
That means I constructed my minimal example poorly, since that's not the case in the real-world issue I'm having.
This one demonstrates the issue I'm having better:
import pytest
@pytest.mark.parametrize(
'bar', [
'abc-1',
'def-1',
'ghi-1',
'jkl-1'
],
)
def test_foo(bar):
assert(True)[kenny@pc test]$ pytest -v -p no:sugar -k 'abc-1 and jkl-1'
============================== test session starts ==============================
platform linux -- Python 3.6.4, pytest-3.4.1, py-1.5.2, pluggy-0.6.0 -- /usr/bin/python
cachedir: .pytest_cache
rootdir: /home/kenny/test, inifile:
plugins: testinfra-1.10.1, xdist-1.22.1, interactive-0.1.4, forked-0.2, pylama-7.4.3
collected 4 items
test_foo.py::test_foo[def-1] PASSED [ 50%]
test_foo.py::test_foo[ghi-1] PASSED [100%]
============================== 2 tests deselected ===============================
==================== 2 passed, 2 deselected in 0.01 seconds =====================
[kenny@pc test]$ pytest -v -p no:sugar -k 'abc-1 and ghi-1'
============================== test session starts ==============================
platform linux -- Python 3.6.4, pytest-3.4.1, py-1.5.2, pluggy-0.6.0 -- /usr/bin/python
cachedir: .pytest_cache
rootdir: /home/kenny/test, inifile:
plugins: testinfra-1.10.1, xdist-1.22.1, interactive-0.1.4, forked-0.2, pylama-7.4.3
collected 4 items
test_foo.py::test_foo[def-1] PASSED [ 50%]
test_foo.py::test_foo[jkl-1] PASSED [100%]
============================== 2 tests deselected ===============================
==================== 2 passed, 2 deselected in 0.01 seconds =====================
[kenny@pc test]$ pytest -v -p no:sugar -k 'abc-1 or ghi-1'
============================== test session starts ==============================
platform linux -- Python 3.6.4, pytest-3.4.1, py-1.5.2, pluggy-0.6.0 -- /usr/bin/python
cachedir: .pytest_cache
rootdir: /home/kenny/test, inifile:
plugins: testinfra-1.10.1, xdist-1.22.1, interactive-0.1.4, forked-0.2, pylama-7.4.3
collected 4 items
test_foo.py::test_foo[abc-1] PASSED [ 25%]
test_foo.py::test_foo[def-1] PASSED [ 50%]
test_foo.py::test_foo[ghi-1] PASSED [ 75%]
test_foo.py::test_foo[jkl-1] PASSED [100%]
=========================== 4 passed in 0.01 seconds ============================Take out the dashes from the parameters and filters and it behaves as expected:
[kenny@pc test]$ pytest -v -p no:sugar -k 'abc1 and ghi1'
============================== test session starts ==============================
platform linux -- Python 3.6.4, pytest-3.4.1, py-1.5.2, pluggy-0.6.0 -- /usr/bin/python
cachedir: .pytest_cache
rootdir: /home/kenny/test, inifile:
plugins: testinfra-1.10.1, xdist-1.22.1, interactive-0.1.4, forked-0.2, pylama-7.4.3
collected 4 items
============================== 4 tests deselected ===============================
========================= 4 deselected in 0.01 seconds ==========================
[kenny@pc test]$ pytest -v -p no:sugar -k 'abc1 or ghi1'
============================== test session starts ==============================
platform linux -- Python 3.6.4, pytest-3.4.1, py-1.5.2, pluggy-0.6.0 -- /usr/bin/python
cachedir: .pytest_cache
rootdir: /home/kenny/test, inifile:
plugins: testinfra-1.10.1, xdist-1.22.1, interactive-0.1.4, forked-0.2, pylama-7.4.3
collected 4 items
test_foo.py::test_foo[abc1] PASSED [ 50%]
test_foo.py::test_foo[ghi1] PASSED [100%]
============================== 2 tests deselected ===============================
==================== 2 passed, 2 deselected in 0.01 seconds =====================I've found elsewhere in this issue tracker that filters are treated as Python expressions. That probably means that the dash is interpreted as a minus sign instead of a literal character.
Unfortunately I really need those dashes in my parameters and I need to be able to filter on them.
rasschaert
commented
6 years ago Meanwhile I've found an ugly workaround that allows me to apply the filters I want.
Replacing the dashes in with underscores in the ids and then also using underscores in the filter works well, without actually changing the original parameter that is fed into my test.
import pytest
@pytest.mark.parametrize(
'bar', [
'abc-1',
'def-1',
'ghi-1',
'jkl-1'
],
ids=lambda x: x.replace('-', '_')
)
def test_foo(bar):
assert("-" in bar and "_" not in bar)[kenny@pc test]$ pytest -v -p no:sugar -k 'abc_1 or jkl_1'
============================== test session starts ==============================
platform linux -- Python 3.6.4, pytest-3.4.1, py-1.5.2, pluggy-0.6.0 -- /usr/bin/python
cachedir: .pytest_cache
rootdir: /home/kenny/test, inifile:
plugins: testinfra-1.10.1, xdist-1.22.1, interactive-0.1.4, forked-0.2, pylama-7.4.3
collected 4 items
test_foo.py::test_foo[abc_1] PASSED [ 50%]
test_foo.py::test_foo[jkl_1] PASSED [100%]
============================== 2 tests deselected ===============================
==================== 2 passed, 2 deselected in 0.01 seconds =====================@rasschaert thans for showing your solution , its a fine hack to work around that current shortcoming of pytest
we hope to eventually enable better selection fo marks, but thats still far off
nicoddemus
commented
6 years ago Closing as this has not seen activity in awhile.
RonnyPfannschmidt
commented
6 years ago @nicoddemus this one hasnt been fixed as far as i can tell
nicoddemus
commented
6 years ago Thanks @RonnyPfannschmidt for catching my slip up.
nicoddemus
commented
6 years ago Looking again at this, I'm not sure if this is solvable: -k matches against mark names as well, so this is a "feature".
If we want to change how -k works (personally I'm not sure if it is worth the trouble), should we create a new proposal ticket for it?
RonnyPfannschmidt
commented
6 years ago personally i wonder if we should just introduce a new option to express filters and then at a later point deprecate -m and -k
Filter expressions (-k) become very unreliable when you try to match the parameters in parameterized fixtures. The filters seem to match things they shouldn't.
Minimal example:
Filtering for 'z' matches everything (which is, I'm pretty sure, incorrect behaviour).
Different filters do work as expected.
My system is Arch Linux, the relevant software versions are in my pytest output. pip list:
Being able to select/deselect certain tests based on the parameter names is quite important to my use case. Please let me know if I can further help narrow down the issue.