pytest-dev / pytest

The pytest framework makes it easy to write small tests, yet scales to support complex functional testing
https://pytest.org
MIT License
11.87k stars 2.65k forks source link

pytest.config.getoption no longner supported on 3.8.2 #4160

Closed AaronV77 closed 5 years ago

AaronV77 commented 5 years ago

Hello,

Issue Summary: I've ran into the issue of the following line pytest.config.getoption no longer being supported in the 3.8.2 conda package of pytest. So the issue that I am having is how to transition from using the following below to using the standard request.config.getoption("--skip-slow"). I'm seeing a lot of documentation inconsistencies and don't know how to go about fixing this; other than:

Thanks for any help!

#conftest.py
def pytest_addoption(parser):
    parser.addoption('--skip-slow', action='store_true')

#test_catalog.py
import pytest

from quest.static import GeomType
from data import SERVICES_CATALOG_COUNT, CACHED_SERVICES

slow = pytest.mark.skipif(
    pytest.config.getoption("--skip-slow"),
    reason="--skip-slow option was set"
)

@slow
def test_add_datasets(api, catalog_entry):
    b = api.add_datasets('col1', catalog_entry)
    c = api.get_datasets(filters={'collection': 'col1'})
    assert len(list(c)) == 1
    assert b == c

Environment Listing:

# Name                    Version                   Build  Channel
affine                    2.2.1                      py_0    conda-forge
alabaster                 0.7.11                     py_3    conda-forge
appdirs                   1.4.3                      py_1    conda-forge
appnope                   0.1.0                    py36_0    conda-forge
asn1crypto                0.24.0                   py36_3    conda-forge
atomicwrites              1.2.1                    py36_0    conda-forge
attrs                     18.2.0                     py_0    conda-forge
babel                     2.6.0                      py_1    conda-forge
backcall                  0.1.0                      py_0    conda-forge
beautifulsoup4            4.6.3                    py36_0    conda-forge
blas                      1.0                         mkl  
bleach                    1.5.0                    py36_0    conda-forge
blinker                   1.4                        py_1    conda-forge
blosc                     1.14.4               hfc679d8_0    conda-forge
bokeh                     1.0.0dev6          pyhdd78535_0    pyviz/label/dev
boost-cpp                 1.67.0               h3a22d5f_0    conda-forge
boto3                     1.9.2                      py_0    conda-forge
botocore                  1.12.2                     py_0    conda-forge
bottleneck                1.2.1            py36h7eb728f_1    conda-forge
bzip2                     1.0.6                         1    conda-forge
ca-certificates           2018.8.24            ha4d7672_0    conda-forge
cairo                     1.14.12              he6fea26_5    conda-forge
cartopy                   0.16.0           py36h81b52dc_2    conda-forge
certifi                   2018.8.24             py36_1001    conda-forge
cffi                      1.11.5           py36h5e8e0c9_1    conda-forge
cftime                    1.0.1            py36h7eb728f_0    conda-forge
chardet                   3.0.4                    py36_3    conda-forge
click                     6.7                        py_1    conda-forge
click-plugins             1.0.3                      py_1    conda-forge
cligj                     0.4.0                      py_1    conda-forge
cloudpickle               0.5.5                      py_0    conda-forge
colorcet                  1.0.0                      py_0    conda-forge
coverage                  4.5.1            py36h470a237_1    conda-forge
coveralls                 1.5.0                      py_0    conda-forge
cryptography              2.3.1            py36hdffb7b8_0    conda-forge
cryptography-vectors      2.3.1                    py36_0    conda-forge
curl                      7.61.0               h93b3f91_2    conda-forge
cycler                    0.10.0                     py_1    conda-forge
cytoolz                   0.9.0.1          py36h470a237_0    conda-forge
dask                      0.19.1                     py_0    conda-forge
dask-core                 0.19.1                     py_0    conda-forge
datashader                0.6.8                      py_0    pyviz/label/dev
datashape                 0.5.4                    py36_0    conda-forge
decorator                 4.3.0                      py_0    conda-forge
descartes                 1.1.0                      py_2    conda-forge
diskcache                 3.0.6                      py_0    conda-forge
distributed               1.23.1                   py36_0    conda-forge
docopt                    0.6.2                      py_1    conda-forge
docutils                  0.14                     py36_1    conda-forge
entrypoints               0.2.3                    py36_2    conda-forge
expat                     2.2.5                hfc679d8_2    conda-forge
fiona                     1.7.13           py36hb00a9d7_3    conda-forge
fontconfig                2.13.0               hce039c3_6    conda-forge
freetype                  2.9.1                h6debe1e_4    conda-forge
freexl                    1.0.5                h470a237_2    conda-forge
freezegun                 0.3.10                   py36_0    conda-forge
future                    0.16.0                   py36_2    conda-forge
futures-compat            1.0                       py3_0    conda-forge
gdal                      2.2.4            py36hb00a9d7_9    conda-forge
geojson                   2.3.0                      py_0    conda-forge
geopandas                 0.4.0                      py_1    conda-forge
geos                      3.6.2                hfc679d8_3    conda-forge
geotiff                   1.4.2                hb54a4aa_4    conda-forge
geoviews                  1.5.4a6                    py_0    pyviz/label/dev
geoviews-core             1.5.4a6                    py_0    pyviz/label/dev
gettext                   0.19.8.1             h1f1d5ed_1    conda-forge
giflib                    5.1.4                h470a237_1    conda-forge
girder-client             2.4.0                      py_0    conda-forge
glib                      2.55.0               h464dc38_2    conda-forge
h5netcdf                  0.6.2                      py_0    conda-forge
h5py                      2.8.0            py36hb794570_1    conda-forge
hdf4                      4.2.13               h951d187_2    conda-forge
hdf5                      1.10.2               hc401514_2    conda-forge
heapdict                  1.0.0                    py36_0    conda-forge
holoviews                 1.11.0a4                   py_0    pyviz/label/dev
hs_restclient             1.2.12                     py_0    conda-forge
html5lib                  0.9999999                py36_0    conda-forge
hvplot                    0.2.1                      py_1    conda-forge
icu                       58.2                 hfc679d8_0    conda-forge
idna                      2.7                      py36_2    conda-forge
imageio                   2.3.0                      py_1    conda-forge
imagesize                 1.1.0                      py_0    conda-forge
intel-openmp              2018.0.3                      0  
ipykernel                 4.9.0                    py36_0    conda-forge
ipython                   6.5.0                    py36_0    conda-forge
ipython_genutils          0.2.0                      py_1    conda-forge
ipywidgets                7.4.1                      py_0    conda-forge
isodate                   0.6.0                      py_1    conda-forge
jedi                      0.12.1                   py36_0    conda-forge
jinja2                    2.10                       py_1    conda-forge
jmespath                  0.9.3                      py_1    conda-forge
jpeg                      9c                   h470a237_1    conda-forge
json-c                    0.12.1               h470a237_1    conda-forge
jsonschema                2.6.0                    py36_2    conda-forge
jupyter                   1.0.0                      py_1    conda-forge
jupyter_client            5.2.3                      py_1    conda-forge
jupyter_console           5.2.0                    py36_1    conda-forge
jupyter_core              4.4.0                      py_0    conda-forge
kealib                    1.4.9                h0bee7d0_2    conda-forge
kiwisolver                1.0.1            py36h2d50403_2    conda-forge
krb5                      1.14.6                        0    conda-forge
libdap4                   3.19.1               h18059cb_1    conda-forge
libffi                    3.2.1                hfc679d8_5    conda-forge
libgdal                   2.2.4                hdb23ba1_9    conda-forge
libgfortran               3.0.0                         1    conda-forge
libiconv                  1.15                 h470a237_3    conda-forge
libkml                    1.3.0                hccc92b1_8    conda-forge
libnetcdf                 4.6.1                h039f2a5_8    conda-forge
libpng                    1.6.35               ha92aebf_2    conda-forge
libpq                     9.6.3                         0    conda-forge
libsodium                 1.0.16               h470a237_1    conda-forge
libspatialindex           1.8.5                hfc679d8_3    conda-forge
libspatialite             4.3.0a              h3b29d86_23    conda-forge
libssh2                   1.8.0                h5b517e9_2    conda-forge
libtiff                   4.0.9                he6b73bb_2    conda-forge
libuuid                   1.0.3                         1    conda-forge
libxml2                   2.9.8                h422b904_4    conda-forge
libxslt                   1.1.32               h88dbc4e_2    conda-forge
llvmlite                  0.23.0                   py36_1    conda-forge
locket                    0.2.0                      py_2    conda-forge
lxml                      4.2.5            py36hc9114bc_0    conda-forge
markupsafe                1.0              py36h470a237_1    conda-forge
matplotlib                2.2.3            py36h0e0179f_0    conda-forge
mistune                   0.8.3            py36h470a237_2    conda-forge
mkl                       2017.0.4             h1fae6ae_0  
mock                      2.0.0                    py36_0    conda-forge
more-itertools            4.3.0                    py36_0    conda-forge
msgpack-python            0.5.6            py36h2d50403_2    conda-forge
multipledispatch          0.6.0                      py_0    conda-forge
munch                     2.3.2                      py_0    conda-forge
nbconvert                 5.3.1                      py_1    conda-forge
nbformat                  4.4.0                      py_1    conda-forge
nbval                     0.9.1                      py_0    conda-forge
ncurses                   6.1                  hfc679d8_1    conda-forge
netcdf4                   1.4.1            py36h62672b6_0    conda-forge
networkx                  2.1                        py_1    conda-forge
notebook                  5.6.0                    py36_1    conda-forge
numba                     0.38.1                   py36_0    conda-forge
numexpr                   2.6.6                    py36_0    conda-forge
numpy                     1.13.1                   py36_0  
oauthlib                  2.1.0                      py_0    conda-forge
olefile                   0.46                       py_0    conda-forge
openblas                  0.2.20                        8    conda-forge
openjpeg                  2.3.0                h316dc23_3    conda-forge
openssl                   1.0.2p               h470a237_1    conda-forge
owslib                    0.16.0                     py_1    conda-forge
packaging                 17.1                       py_0    conda-forge
pandas                    0.22.0                   py36_1    conda-forge
pandoc                    2.2.2                hde52d81_1    conda-forge
pandocfilters             1.4.2                      py_1    conda-forge
param                     1.8.0a2                    py_0    pyviz/label/dev
parambokeh                0.2.3                      py_0    pyviz/label/dev
paramnb                   2.0.3                         0    ioam
parso                     0.3.1                      py_0    conda-forge
partd                     0.3.8                      py_1    conda-forge
pbr                       4.2.0                      py_0    conda-forge
pcre                      8.41                 hfc679d8_3    conda-forge
peewee                    3.5.2            py36h65ede16_0    conda-forge
pexpect                   4.6.0                    py36_0    conda-forge
pickleshare               0.7.4                    py36_0    conda-forge
pillow                    5.2.0            py36hc736899_1    conda-forge
pint                      0.8.1                      py_1    conda-forge
pip                       18.0                     py36_1    conda-forge
pixman                    0.34.0               h470a237_3    conda-forge
plotly                    3.2.1                      py_0    conda-forge
pluggy                    0.7.1                      py_0    conda-forge
pony                      0.7.1                    py36_0    conda-forge
poppler                   0.67.0               h4d7e492_3    conda-forge
poppler-data              0.4.9                         0    conda-forge
postgresql                10.5             py36h286b696_0    conda-forge
proj4                     4.9.3                h470a237_8    conda-forge
prometheus_client         0.3.1                      py_1    conda-forge
prompt_toolkit            1.0.15                     py_1    conda-forge
psutil                    5.4.7            py36h470a237_1    conda-forge
psycopg2                  2.7.5            py36hdffb7b8_1    conda-forge
ptyprocess                0.6.0                    py36_0    conda-forge
py                        1.6.0                      py_0    conda-forge
pycparser                 2.18                       py_1    conda-forge
pyct                      0.4.5                      py_0    pyviz/label/dev
pyct-core                 0.4.5                      py_0    pyviz/label/dev
pyepsg                    0.3.2                      py_1    conda-forge
pygments                  2.2.0                      py_1    conda-forge
pyjwt                     1.6.4                      py_0    conda-forge
pyopenssl                 18.0.0                   py36_0    conda-forge
pyparsing                 2.2.0                      py_1    conda-forge
pyproj                    1.9.5.1          py36h508ed2a_3    conda-forge
pyqt                      5.6.0            py36h8210e8a_7    conda-forge
pysal                     1.14.4.post2             py36_0    conda-forge
pyshp                     1.2.12                     py_0    conda-forge
pysocks                   1.6.8                    py36_2    conda-forge
pytables                  3.4.4            py36h247b57e_1    conda-forge
pytest                    3.8.2                 py36_1000    conda-forge
pytest-cov                2.6.0                      py_0    conda-forge
pytest-runner             4.2                        py_1    conda-forge
python                    3.6.6                h5001a0f_0    conda-forge
python-dateutil           2.7.3                      py_0    conda-forge
pytz                      2018.5                     py_0    conda-forge
pyviz_comms               0.6.0                      py_0    pyviz/label/dev
pywavelets                1.0.0            py36h7eb728f_0    conda-forge
pyyaml                    3.13             py36h470a237_1    conda-forge
pyzmq                     17.1.2           py36hae99301_0    conda-forge
qt                        5.6.2                hd4c90f3_9    conda-forge
qtconsole                 4.4.1                    py36_1    conda-forge
rasterio                  1.0.3            py36h1b5fcde_0    conda-forge
readline                  7.0                  haf1bffa_1    conda-forge
requests                  2.19.1                   py36_1    conda-forge
requests-oauthlib         1.0.0                      py_1    conda-forge
requests-toolbelt         0.8.0                      py_1    conda-forge
retrying                  1.3.3                      py_2    conda-forge
rtree                     0.8.3                    py36_0    conda-forge
s3transfer                0.1.13                   py36_1    conda-forge
scikit-image              0.14.0           py36hfc679d8_1    conda-forge
scipy                     0.19.1              np113py36_0  
send2trash                1.5.0                      py_0    conda-forge
setuptools                40.2.0                   py36_0    conda-forge
shapely                   1.6.4            py36h164cb2d_1    conda-forge
simplegeneric             0.8.1                      py_1    conda-forge
sip                       4.18.1           py36hfc679d8_0    conda-forge
six                       1.11.0                   py36_1    conda-forge
snowballstemmer           1.2.1                      py_1    conda-forge
snuggs                    1.4.1                      py_1    conda-forge
sortedcollections         1.0.1                      py_1    conda-forge
sortedcontainers          2.0.4                      py_1    conda-forge
sphinx                    1.7.9                    py36_0    conda-forge
sphinx_rtd_theme          0.4.1                      py_0    conda-forge
sphinxcontrib-websupport  1.1.0                      py_1    conda-forge
sqlalchemy                1.2.11           py36h470a237_0    conda-forge
sqlite                    3.24.0               h2f33b56_1    conda-forge
suds-jurko                0.6                      py36_1    conda-forge
tblib                     1.3.2                      py_1    conda-forge
terminado                 0.8.1                    py36_1    conda-forge
testpath                  0.3.1                    py36_1    conda-forge
tk                        8.6.8                         0    conda-forge
toolz                     0.9.0                      py_0    conda-forge
tornado                   5.1              py36h470a237_1    conda-forge
traitlets                 4.3.2                    py36_0    conda-forge
ulmo                      0.8.4                      py_1    conda-forge
urllib3                   1.23                     py36_1    conda-forge
wcwidth                   0.1.7                      py_1    conda-forge
webencodings              0.5.1                      py_1    conda-forge
werkzeug                  0.14.1                     py_0    conda-forge
wheel                     0.31.1                   py36_1    conda-forge
whitebox_tools            0.8.0dev                 py36_0    conda-forge
widgetsnbextension        3.4.1                    py36_0    conda-forge
xarray                    0.10.8                   py36_0    conda-forge
xerces-c                  3.2.0                h5d6a6da_2    conda-forge
xz                        5.2.4                h470a237_1    conda-forge
yaml                      0.1.7                h470a237_1    conda-forge
zeromq                    4.2.5                hfc679d8_5    conda-forge
zict                      0.1.3                      py_0    conda-forge
zlib                      1.2.11               h470a237_3    conda-forge

Operating System Version:

nicoddemus commented 5 years ago

Hi @AaronV77,

I think the easiest way to accomplish the same thing is to create your own mark, say pytest.mark.slow:

@pytest.mark.slow
def test_add_datasets(api, catalog_entry):
    ...

And then you can skip those tests by running -m "not slow", no additional code required.

AaronV77 commented 5 years ago

@nicoddemus Thank you for the quick response. Is there anyway to provide more than one argument for the -m option? Lets say if I have a two groups of test to run, I've tried the following:

Neither of these work, and possible way? Thanks

nicoddemus commented 5 years ago

You can use and, not, and or operators: pytest test -s -m "slow and download"

AaronV77 commented 5 years ago

@nicoddemus just actually found that! When I run it though it deselects those two test, instead of only running those two tests?

nicoddemus commented 5 years ago

It selects the tests with those marks. If you don't want to run the tests, you can use not:

pytest test -s -m "not slow and not download"
AaronV77 commented 5 years ago

I know that, but what I'm trying to say is that the following command pytest test -s -m "slow and download" is not in fact running the test with the selected marks.

import pytest

pytest.mark.print
def test_printing():
    print("Something-1")

@pytest.mark.print2
def test_printing2():
    print("Something-2")

def test_printing3():
    print("Something-3")

def test_printing4():
    print("Something-4")

Command pytest test -s:

(quest-py3.6-DEV) AaronV77:pytest AaronV77$ pytest test -s
==========================================================================test session starts ==========================================================================
platform darwin -- Python 3.6.6, pytest-3.8.2, py-1.6.0, pluggy-0.7.1
rootdir: /Users/AaronV77/Documents/pytest, inifile: pytest.ini
plugins: cov-2.6.0, nbval-0.9.1
collected 4 items                                                                                                                                                                                                                                                                         

test/test_anything.py Something-1
.Something-2
.Something-3
.Something-4
.
==========================================================================4 passed in 0.02 seconds ==========================================================================

Command pytest test -s -m print:

(quest-py3.6-DEV) AaronV77:pytest AaronV77$ pytest test -s -m print
==========================================================================test session starts ==========================================================================
platform darwin -- Python 3.6.6, pytest-3.8.2, py-1.6.0, pluggy-0.7.1
rootdir: /Users/AaronV77/Documents/pytest, inifile: pytest.ini
plugins: cov-2.6.0, nbval-0.9.1
collected 4 items / 3 deselected                                                                                                                                                                                                                                                          

test/test_anything.py Something-1
.
==========================================================================1 passed, 3 deselected in 0.01 seconds ==========================================================================

Command pytest test -s -m print2:

(quest-py3.6-DEV)AaronV77:pytest AaronV77$ pytest test -s -m print2
========================================================================== test session starts ==========================================================================
platform darwin -- Python 3.6.6, pytest-3.8.2, py-1.6.0, pluggy-0.7.1
rootdir: /Users/AaronV77/Documents/pytest, inifile: pytest.ini
plugins: cov-2.6.0, nbval-0.9.1
collected 4 items / 3 deselected                                                                                                                                                                                                                                                          

test/test_anything.py Something-2
.
==========================================================================1 passed, 3 deselected in 0.01 seconds ==========================================================================

Command `pytest test -s -m "print and print2':

(quest-py3.6-DEV) AaronV77:pytest AaronV77$ pytest test -s -m "print and print2"
==========================================================================test session starts ==========================================================================
platform darwin -- Python 3.6.6, pytest-3.8.2, py-1.6.0, pluggy-0.7.1
rootdir: /Users/AaronV77/Documents/pytest, inifile: pytest.ini
plugins: cov-2.6.0, nbval-0.9.1
collected 4 items / 4 deselected                                                                                                                                                                                                                                                          
==========================================================================4 deselected in 0.04 seconds ==========================================================================

Command `pytest test -s -m "not print and not print2':

(quest-py3.6-DEV) AaronV77:pytest AaronV77$ pytest test -s -m "not print and not print2"
==========================================================================test session starts ==========================================================================
platform darwin -- Python 3.6.6, pytest-3.8.2, py-1.6.0, pluggy-0.7.1
rootdir: /Users/AaronV77/Documents/pytest, inifile: pytest.ini
plugins: cov-2.6.0, nbval-0.9.1
collected 4 items / 2 deselected                                                                                                                                                                                                                                                          

test/test_anything.py Something-3
.Something-4
.
==========================================================================2 passed, 2 deselected in 0.01 seconds ==========================================================================
AaronV77 commented 5 years ago

@nicoddemus Never mind I understand now how the and, and the or works. In my mind I thought it would run all the tests that were marked with either print or print2, but in reality the test is only ran if it is marked with both print and print2. Sorry for the confusion.

ssbarnea commented 5 years ago

Can one of you give me a hint on how to fix that issue on https://github.com/ansible/molecule/blob/master/test/conftest.py#L114 ? Thanks.

nicoddemus commented 5 years ago

@ssbarnea sorry for the delay. Can you provide a link for a failing build?