nodev-io / pytest-nodev

Test-driven source code search for Python.
http://pytest-nodev.readthedocs.io
MIT License
27 stars 8 forks source link

Modules, functions and classes crashing or hanging to be added to the blacklist. #12

Open alexamici opened 8 years ago

alexamici commented 8 years ago

pytest-nodev have a module blacklist and an object blacklist that is update with every release.

If the enabling pytest-nodev crashes of hangs the pytest process you have probably found a problem object that needs to be added to the blacklist. In order to identify the problem object remove all non essential options from the command line, in particular --candidates-fail and -n, and add -vv instead. Any test line that doesn't end with xfail is a good candidate, especially if it is the last line before the summary or the crash report. For example the following command hangs:

$ py.test -vv examples/test_rfc3986_parse.py --candidates-from-modules itertools
[...]
examples/test_rfc3986_parse.py::test_rfc3986_parse_basic[itertools:count] xfail
examples/test_rfc3986_parse.py::test_rfc3986_parse_full[itertools:count] xfail
examples/test_rfc3986_parse.py::test_rfc3986_parse_basic[itertools:cycle]

and this is a hint that itertools:cycle causes the hang.

Comment here to submit objects that crash or hang during the candidate fixture for review, be sure to mention the the full object name, usually the string in square brackets.

francesconazzaro commented 8 years ago

the following modules and functions hang pytest process:

kr1 commented 8 years ago

pycallgraph crashes pytest-wish because it monkeypatches inspect.getmodule at import.
This is therefore a general case (albeit probably infrequent). A simple workaround would be a contextmanager around those calls, along the lines of:

+@contextmanager
+def reset_function(function):
+    original_function = function
+    yield
+    function = original_function

@@ -104,13 +110,15 @@ def import_distributions(distribution_modules,           
module_blacklist_pattern=MODULE_B

def generate_module_objects(module, predicate=None):
    try:
-        module_members = inspect.getmembers(module, predicate)
+        with reset_function(getmembers):
+            module_members = getmembers(module, predicate)
alexamici commented 8 years ago

@ kr1 I don't think we can really protect from modules that fiddle with the infrastructure that we use, mainly module load and introspection, as we would need to save and restore also all nested functions. The only safe strategies are:

Good catch! I added it to the blacklist for now.

alexamici commented 8 years ago

Blacklisted many stdlib objects with broken __del__ methods that pollute output with Exception ignored in: noise.

Move issue to next release.