pytest-dev / pytest

The pytest framework makes it easy to write small tests, yet scales to support complex functional testing
https://pytest.org
MIT License
11.87k stars 2.64k forks source link

filterwarnings does not work with request.applymarker #5677

Closed spumer closed 4 years ago

spumer commented 5 years ago

I whant to manage filterwarnings from shared fixture, for example filter urllib3.exceptions.InsecureRequestWarning when requests client fixture used.

Example:

@pytest.fixture()
def raw_client(request):
    request.applymarker(pytest.mark.filterwarnings('ignore::urllib3.exceptions.InsecureRequestWarning'))
    return RequestsClient()

def test_oauth_error(raw_client, oauth_error):
    resp = raw_client.get('https://testserver/auth/obtain-token', params={'code': '123'})
    assert resp.status_code == 400, resp.json()

Above usage not worked for me, but filtering through pytest.ini works fine:

filterwarnings =
    ignore::urllib3.exceptions.InsecureRequestWarning
$ pip list
Package                          Version  
-------------------------------- ---------
alabaster                        0.7.12   
amqp                             2.5.0    
aspy.yaml                        1.3.0    
atomicwrites                     1.3.0    
attrs                            19.1.0   
Babel                            2.7.0    
billiard                         3.5.0.5  
celery                           4.1.1    
certifi                          2019.6.16
cfgv                             2.0.1    
chardet                          3.0.4    
CommonMark                       0.5.4    
coreapi                          2.3.3    
coreschema                       0.0.4    
Django                           2.1.7    
django-filter                    2.1.0    
django-model-utils               3.2.0    
django-oauth2-login-client       0.8.0    
django-postgrespool2             0.2.0    
djangorestframework              3.9.2    
djangorestframework-role-filters 0.3.0    
docutils                         0.15     
drf-yasg                         1.15.0   
elementpath                      1.1.8    
environs                         4.1.0    
factory-boy                      2.11.1   
Faker                            2.0.0    
freezegun                        0.3.11   
gunicorn                         19.9.0   
identify                         1.4.5    
idna                             2.8      
imagesize                        1.1.0    
importlib-metadata               0.18     
inflection                       0.3.1    
itypes                           1.1.0    
Jinja2                           2.10.1   
json-rpc                         1.11.0   
kombu                            4.2.1    
lxml                             4.3.4    
MarkupSafe                       1.1.1    
marshmallow                      2.19.5   
more-itertools                   7.2.0    
multidict                        4.5.2    
nodeenv                          1.3.3    
oauthlib                         3.0.1    
packaging                        19.0     
pip                              19.2     
pluggy                           0.12.0   
pre-commit                       1.15.1   
psycopg2-binary                  2.7.7    
py                               1.8.0    
Pygments                         2.4.2    
pyparsing                        2.4.1    
pytest                           5.0.1    
pytest-django                    3.5.1    
pytest-env                       0.6.2    
pytest-freezegun                 0.3.0    
pytest-mock                      1.10.0   
pytest-testdox                   1.1.1    
pytest-timeout                   1.3.3    
python-dateutil                  2.8.0    
python-dotenv                    0.10.3   
pytz                             2019.1   
PyYAML                           5.1.1    
raven                            6.10.0   
recommonmark                     0.4.0    
requests                         2.21.0   
requests-mock                    1.5.2    
requests-oauthlib                1.2.0    
ruamel.yaml                      0.15.100 
setuptools                       41.0.1   
simplejson                       3.16.0   
six                              1.12.0   
slackclient                      1.3.1    
snowballstemmer                  1.9.0    
Sphinx                           1.8.1    
sphinx-rtd-theme                 0.4.2    
sphinxcontrib-websupport         1.1.2    
SQLAlchemy                       1.3.6    
text-unidecode                   1.2      
toml                             0.10.0   
trafaret                         1.2.0    
uritemplate                      3.0.0    
urllib3                          1.24.3   
vine                             1.3.0    
virtualenv                       16.7.0   
wcwidth                          0.1.7    
websocket-client                 0.54.0   
wheel                            0.33.4   
xmldiff                          2.3      
xmlschema                        1.0.13   
xmltodict                        0.12.0   
yarl                             1.3.0    
zipp                             0.5.2 
RonnyPfannschmidt commented 5 years ago

At fixture set up time warning is already configured, you need a imperative way to configure your warnings, I'm on mobile, so I won't find the docs links

nicoddemus commented 5 years ago

As @RonnyPfannschmidt said, at fixture setup time pytest's own warnings were already configured, so pytest's internal warning mechanisms never has a chance of "seeing" your filter.

I suggest this (untested):

@pytest.fixture()
def raw_client(request):
    return RequestsClient()

def test_oauth_error(raw_client, oauth_error):
    with warnings.catch_warnings():
        resp = raw_client.get('https://testserver/auth/obtain-token', params={'code': '123'})
        assert resp.status_code == 400, resp.json()
spumer commented 5 years ago

As @RonnyPfannschmidt said, at fixture setup time pytest's own warnings were already configured, so pytest's internal warning mechanisms never has a chance of "seeing" your filter.

I suggest this (untested):

@pytest.fixture()
def raw_client(request):
    return RequestsClient()

def test_oauth_error(raw_client, oauth_error):
    with warnings.catch_warnings():
        resp = raw_client.get('https://testserver/auth/obtain-token', params={'code': '123'})
        assert resp.status_code == 400, resp.json()

Yea, but i whant to disable warnings explicit, not for all. Your case can be rewritten by mark: https://docs.pytest.org/en/latest/warnings.html#pytest-mark-filterwarnings

And this why i expect applymark will work fine https://docs.pytest.org/en/latest/reference.html#_pytest.fixtures.FixtureRequest.applymarker

adammrozik commented 3 years ago

Unlocking this subject, as I have similar issue.

I have a custom pytest plugin which creates infrastructure, which I mark with @infra(name ='postgres'). However, I want to skip tests if the creation of the infrastructure fails.

Right now I cannot do it, because if I do applymarkers or add_marker inside a fixture, it is already too late. Is there a way to do it?