search

pytest-dev / pytest

The pytest framework makes it easy to write small tests, yet scales to support complex functional testing
https://pytest.org
MIT License
12.02k stars 2.67k forks source link

applymarker doesn`t work for custom marks #7540

Closed simpletonDL closed 4 years ago

simpletonDL commented 4 years ago

Hello, I have parameterized fixture and I want to mark function invocaions differently. So, for example:

import pytest

@pytest.fixture(params=[1, 2, 3])
def some_fixture(request):
    if request.param == 2:
        request.applymarker(pytest.mark.slow)
    return request.param

def test_something(some_fixture):
    pass

But when I try to run marked test function, pytest doesn`t find them:

========================================= test session starts=========================================
platform linux -- Python 3.7.5, pytest-5.4.3, py-1.9.0, pluggy-0.13.1
benchmark: 3.2.3 (defaults: timer=time.perf_counter disable_gc=False min_rounds=5 min_time=0.000005 max_time=1.0 calibration_precision=10 warmup=False warmup_iterations=100000)
rootdir: /mnt/c/Users/79128/Desktop/Repo/CFPQ_PyAlgo, inifile: pytest.ini
plugins: benchmark-3.2.3, cov-2.10.0
collected 3 items / 3 deselected
======================================== 3 deselected in0.03s========================================

The solution that uses pytest.mark.parametrize decorator won`t suite me, because I need to get parameters in runtime. The second question is how to create markers using python string? So, I want to achive something like pytest.mark.gen_mark(str). My installed packages:

asn1crypto (0.24.0)
attrs (19.3.0)
Automat (0.6.0)
backcall (0.2.0)
bleach (3.1.5)
blinker (1.4)
certifi (2018.1.18)
cffi (1.14.0)
cfpq-data-devtools (1.0.0)
chardet (3.0.4)
click (6.7)
cloud-init (19.4)
colorama (0.4.3)
command-not-found (0.3)
configobj (5.0.6)
constantly (15.1.0)
coverage (5.2)
cryptography (2.1.4)
decorator (4.4.2)
defusedxml (0.6.0)
distro-info (0.18ubuntu0.18.04.1)
entrypoints (0.3)
graphviz (0.14.1)
httplib2 (0.9.2)
hyperlink (17.3.1)
idna (2.6)
importlib-metadata (1.7.0)
incremental (16.10.1)
ipdb (0.13.3)
ipykernel (5.3.2)
ipython (7.16.1)
ipython-genutils (0.2.0)
isodate (0.6.0)
jedi (0.17.1)
Jinja2 (2.11.2)
jsonpatch (1.16)
jsonpointer (1.10)
jsonschema (3.2.0)
jupyter-client (6.1.6)
jupyter-core (4.6.3)
keyring (10.6.0)
keyrings.alt (3.0)
language-selector (0.1)
llvmlite (0.33.0)
MarkupSafe (1.1.1)
mistune (0.8.4)
more-itertools (8.4.0)
nbconvert (5.6.1)
nbformat (5.0.7)
netifaces (0.10.4)
networkx (2.4)
notebook (6.0.3)
numba (0.50.1)
numpy (1.19.0)
oauthlib (2.0.6)
packaging (20.4)
PAM (0.4.2)
pandas (1.0.5)
pandocfilters (1.4.2)
parso (0.7.0)
pexpect (4.8.0)
pickleshare (0.7.5)
pip (9.0.1)
pluggy (0.13.1)
prometheus-client (0.8.0)
prompt-toolkit (3.0.5)
ptyprocess (0.6.0)
py (1.9.0)
py-cpuinfo (7.0.0)
pyasn1 (0.4.2)
pyasn1-modules (0.2.1)
pycparser (2.20)
pycrypto (2.6.1)
pydot (1.4.1)
pyformlang (0.1.20)
Pygments (2.6.1)
pygobject (3.26.1)
pygraphblas (3.2.0)
PyJWT (1.5.3)
pyOpenSSL (17.5.0)
pyparsing (2.4.7)
pyrsistent (0.16.0)
pyserial (3.4)
pytest (5.4.3)
pytest-benchmark (3.2.3)
pytest-cov (2.10.0)
python-apt (1.6.5+ubuntu0.2)
python-dateutil (2.8.1)
python-debian (0.1.32)
python-louvain (0.14)
pytz (2020.1)
pyxdg (0.25)
PyYAML (3.12)
pyzmq (19.0.1)
rdflib (5.0.0)
requests (2.18.4)
requests-unixsocket (0.1.5)
rise (5.6.1)
scipy (1.5.1)
SecretStorage (2.3.1)
Send2Trash (1.5.0)
service-identity (16.0.0)
setuptools (49.2.0)
six (1.15.0)
ssh-import-id (5.7)
systemd-python (234)
terminado (0.8.3)
testpath (0.4.4)
tornado (6.0.4)
tqdm (4.47.0)
traitlets (4.3.3)
Twisted (17.9.0)
ufw (0.36)
unattended-upgrades (0.1)
urllib3 (1.22)
wcwidth (0.2.5)
webencodings (0.5.1)
wheel (0.30.0)
zipp (3.1.0)
zope.interface (4.3.2)

Thanks in advance for your attention!

RonnyPfannschmidt commented 4 years ago

unfortunately fixtures run after test selection is done, so the markers you want to use for selection get added only after selection

RonnyPfannschmidt commented 4 years ago

but there is a trick i jsut remembered pytest.param(2, marks=pytest.mark.slow) should allow you to set the marker in the parameters (untested, please crosscheck the docs)

simpletonDL commented 4 years ago

Oh, it is interesting! Do you know, can I build array of such parameters in runtime? Exactly, I need to extract some file paths and I don't really want to hardcode them.

RonnyPfannschmidt commented 4 years ago

its possible to do the building of parameter lists in hooks, but it has to happen at collect time

it helps to create a pytest plugin that manages the related resources to both the pytest_generate_tests hook and fixtures can share some objects with metadata

simpletonDL commented 4 years ago

Okey, thank you very much) I try to parametrize tests with os.listdir and it has worked. In future I also will try hook, that you suggested. Can you please tell me how create mark with string (like pyteset.mark.some_function(str)) or it is not possible?

RonnyPfannschmidt commented 4 years ago

im not sure what you mean by that, pytest.mark.something("astring") can work just fine if thats still not clear, please make up a example so that i can better understand what you try

simpletonDL commented 4 years ago

To be honest I am newbie in pytest, so I dont know what exactly means parameter "astring" (as I understand you mark function with something, but I don`t understand the parameter you passed). I want something like getattr(pytest.mark, some_variable_marker), and surprisingly it worked. So I think this isse can be closed. Thank you very much for the answers and quick response!)

RonnyPfannschmidt commented 4 years ago

Thanks, astring ist literally Just a random String without meaning