Disable all third-party warnings

[timokau]

Hi. I'm using pytest 5.2.1 on NixOS. When testing a library, I get a lot of deprecation warnings from third party libraries that I have no control over. Is there some way to set up a warnings filter to only show warnings originating in the module I'm working on?

I was surprised that I couldn't find a pre-existing issue discussing this. Maybe I just didn't search for the right terms. If there already is one, please point me in the right direction.

`pip list`

``` Package Version ------------------ ------------------- aiohttp 3.6.2 aiohttp-cors 0.7.0 appdirs 1.4.3 astroid 2.3.2 async-timeout 3.0.1 atomicwrites 1.3.0 attrs 19.3.0 backcall 0.1.0 backports-abc 0.5 beautifulsoup4 4.8.1 black 19.10b0 Bottleneck 1.2.1 certifi 2019.9.11 cffi 1.13.1 chardet 3.0.4 Click 7.0 coverage 4.5.4 cryptography 2.8 cycler 0.10.0 decorator 4.4.0 Deprecated 1.2.6 dill 0.3.1.1 docutils 0.15.2 ecdsa 0.13.3 et-xmlfile 1.0.1 funcsigs 1.0.2 future 0.18.1 html5lib 1.0.1 httpretty 0.9.7 idna 2.8 importlib-metadata 0.23 ipython 7.8.0 ipython-genutils 0.2.0 isort 4.3.21 jdcal 1.4.1 jedi 0.15.1 joblib 0.14.0 kiwisolver 1.1.0 lazy-object-proxy 1.4.2 lxml 4.4.1 lz4 2.1.10 matplotlib 3.1.1 mccabe 0.6.1 mock 3.0.5 more-itertools 7.2.0 multidict 4.5.2 multiprocess 0.70.9 networkx 2.4 nix-bisect 0.1.0 nose 1.3.7 notmuch 0.29.2 numexpr 2.7.0 numpy 1.17.3 olefile 0.46 openpyxl 3.0.0 packaging 19.2 pandas 0.25.2 parso 0.5.1 path.py 12.0.1 pathspec 0.6.0 pbr 5.4.3 pexpect 4.7.0 pickleshare 0.7.5 Pillow 6.2.1 pip 19.3.1 pluggy 0.13.0 prompt-toolkit 2.0.10 psutil 5.6.3 ptyprocess 0.6.0 py 1.8.0 pyasn1 0.4.7 pycairo 1.18.1 pycallgraph 1.0.1 pycparser 2.19 pydot 1.4.1 PyGithub 1.44 Pygments 2.4.2 PyGObject 3.32.1 pygraphviz 1.5 PyJWT 1.7.1 pylint 2.4.3 pyOpenSSL 19.0.0 pyparsing 2.4.2 PySocks 1.7.0 pytest 5.2.1 python-dateutil 2.8.0 python-jose 3.0.1 pytz 2019.3 r2pipe 1.4.2 regex 2019.8.19 requests 2.22.0 rsa 4.0 scipy 1.3.1 setuptools 41.4.0.post20191022 setuptools-scm 3.3.3 singledispatch 3.4.0.3 six 1.12.0 soupsieve 1.9.4 SQLAlchemy 1.3.10 tables 3.6.1 toml 0.10.0 tornado 5.1 traitlets 4.3.3 typed-ast 1.4.0 urllib3 1.25.6 wcwidth 0.1.7 webencodings 0.5.1 wrapt 1.11.2 xlrd 1.2.0 xlwt 1.3.0 yarl 1.3.0 zipp 0.6.0 ```

[Zac-HD]

Based on the stdlib docs, you could set up a filter that emits all warnings from your module, and then a final filter that ignores everything not yet matched.

[timokau]

I don't think that solves my issue, since that would also silence DeprecationWarnings that are raised by some of our code. Those would not be raised in our module, but triggered by a line in our module. So

import some_library
some_library.some_deprecated_function()

Should raise a warning in pytest, since it is something we should fix.

import other_library
some_library.some_function_that_uses_a_deprecated_function_internally()

Should not raise a warning, since it is something that a third party (other_library) should fix.

[RonnyPfannschmidt]

pytest cant necessarily know the scope of deprecation warnings for your project perhaps a helper to line up things you care about can help

note that deprecations from other projects are potentially a important dependency health indication

[timokau]

pytest cant necessarily know the scope of deprecation warnings for your project

What do you mean by that? The backtrace will show the line which caused the warning, so the information should be available. Or am I missing something?

[RonnyPfannschmidt]

it cant know what packages/name-spaces belong to your project or which ones you need to take deeper care of (we probably can make a guess for the package, but its still potentially wrong)

[timokau]

That could be configurable though, while defaulting either to a heuristic or to "show all warnings". This seems very relevant. Apparently there was a similar discussion for python's default behaviour. There they thought about implementing a similar behaviour as I want, but didn't go through with it because of the need of heuristics.

pytest is in a slightly different position though, since it is configurable.

[aklajnert]

I think it could be done in two ways:

• allow ignoring warnings from modules under site-packages directory,
• allow ignoring warnings from outside a given directory tree (project root).

[timokau]

Both would work, and I think the second option makes most sense (no reason to special-case site-packages).

[RonnyPfannschmidt]

@timokau i have multiple setups where i have multiple packages on editable isntalls and a test run should warn for all of them, so for me having warnings for all editable installs seems helpful, so using site-packages as indicator seems potentially helpful

a key question for me is how to enable that with the python warning system

[aklajnert]

a key question for me is how to enable that with the python warning system

I don't know how the pytest works in the scope of warnings, but patching warnings.warn could do the trick.

[RonnyPfannschmidt]

@aklajnert currently pytest integrates the warning filtering system, i'd like to solve this in terms of that if possible, else its going to add a really strange maintenance burden

[Kache]

I would've thought the most natural way to do this cleanly is config either:

• allowlist of path prefixes (I would set source folders, maybe ignoring test and misc scripts)
• blocklist of path prefixes (I would set my .venv or site-packages dir)

[cjblocker]

I was looking for a solution to this and this thread came up in Google. I was able to do what I needed with the standard lib warning filter in pytest similar to what Zac suggested.

[tool.pytest.ini_options]
filterwarnings = [
    "ignore",
    "default:::mymodule",
]

In my case, I was calling a scipy function that was deprecated, and I was also calling other packages that called this same scipy function. Adding this filter silenced the warnings coming from calls in other packages, but left the warning from where I was calling it in mymodule.

I think this depends on the packages that are emitting the warning setting the stacklevel when they call warnings.warn() so that it points to your code. If they set it too low, you may silence their warning even when it was triggered by your code.

[Zac-HD]

I think this is the best solution available - we don't want to replace the standard library's warnings system, and "ignore warnings not in my code" does inherently rely on your upstream maintainers to handle the warnings in theirs properly.

(Personally, I like to convert all warnings to errors for at least some CI jobs, and explicitly ignore the few things I can't explicitly fix)

[mokeyish]

Third-party warnings are meaningless, and I can't modify them directly.