search

pytest-dev / pytest

The pytest framework makes it easy to write small tests, yet scales to support complex functional testing
https://pytest.org
MIT License
12.03k stars 2.67k forks source link

`pytest` hangs when `RecursionError` happens for `MagicMock` #8482

Open jiasli opened 3 years ago

jiasli commented 3 years ago

A demo for an infinite recursion:

# test_func.py

from unittest import mock
depth = 0

def func(obj):
    global depth
    depth = depth + 1
    print(depth)
    if depth == 1000:
        raise Exception
    return func(obj.attr)  # Infinite loop

def test_func():
    func(mock.MagicMock())

test_func()
$ python test_func.py
...
983
984
985
Traceback (most recent call last):
  File "test_func.py", line 20, in <module>
    test_func()
  ...
    if type(value) is cls:
RecursionError: maximum recursion depth exceeded while calling a Python object

But pytest just hangs:

$ pytest test_func.py --capture no
...
940
941
942

This happens on both Linux and Windows.

> pytest -V
pytest 6.2.2
The-Compiler commented 3 years ago

Looks like it hangs here:

Current thread 0x00007f34402db740 (most recent call first):
  File "/usr/lib/python3.9/unittest/mock.py", line 2436 in __init__
  File "/usr/lib/python3.9/unittest/mock.py", line 1131 in _increment_mock_call
  File "/usr/lib/python3.9/unittest/mock.py", line 1092 in __call__
  File "?", line 1 in <module>
  File ".../pytest/src/_pytest/_code/code.py", line 166 in eval
  File ".../pytest/src/_pytest/_code/code.py", line 425 in recursionindex
  File ".../pytest/src/_pytest/_code/code.py", line 884 in _truncate_recursive_traceback
  File ".../pytest/src/_pytest/_code/code.py", line 851 in repr_traceback
  File ".../pytest/src/_pytest/_code/code.py", line 923 in repr_excinfo
  File ".../pytest/src/_pytest/_code/code.py", line 665 in getrepr
  File ".../pytest/src/_pytest/nodes.py", line 437 in _repr_failure_py
  File ".../pytest/src/_pytest/nodes.py", line 509 in repr_failure
  File ".../pytest/src/_pytest/runner.py", line 395 in pytest_make_collect_report
  File ".../pytest/.venv/lib/python3.9/site-packages/pluggy/callers.py", line 187 in _multicall
  File ".../pytest/.venv/lib/python3.9/site-packages/pluggy/manager.py", line 84 in <lambda>
  File ".../pytest/.venv/lib/python3.9/site-packages/pluggy/manager.py", line 93 in _hookexec
  File ".../pytest/.venv/lib/python3.9/site-packages/pluggy/hooks.py", line 286 in __call__
  File ".../pytest/src/_pytest/runner.py", line 544 in collect_one_node
  File ".../pytest/src/_pytest/main.py", line 820 in genitems
  File ".../pytest/src/_pytest/main.py", line 649 in perform_collect
  File ".../pytest/src/_pytest/main.py", line 336 in pytest_collection
  File ".../pytest/.venv/lib/python3.9/site-packages/pluggy/callers.py", line 187 in _multicall
  File ".../pytest/.venv/lib/python3.9/site-packages/pluggy/manager.py", line 84 in <lambda>
  File ".../pytest/.venv/lib/python3.9/site-packages/pluggy/manager.py", line 93 in _hookexec
  File ".../pytest/.venv/lib/python3.9/site-packages/pluggy/hooks.py", line 286 in __call__
  File ".../pytest/src/_pytest/main.py", line 325 in _main
  File ".../pytest/src/_pytest/main.py", line 272 in wrap_session
  File ".../pytest/src/_pytest/main.py", line 319 in pytest_cmdline_main
  File ".../pytest/.venv/lib/python3.9/site-packages/pluggy/callers.py", line 187 in _multicall
  File ".../pytest/.venv/lib/python3.9/site-packages/pluggy/manager.py", line 84 in <lambda>
  File ".../pytest/.venv/lib/python3.9/site-packages/pluggy/manager.py", line 93 in _hookexec
  File ".../pytest/.venv/lib/python3.9/site-packages/pluggy/hooks.py", line 286 in __call__
  File ".../pytest/src/_pytest/config/__init__.py", line 168 in main
  File ".../pytest/src/_pytest/config/__init__.py", line 191 in console_main
  File ".../pytest/.venv/bin/pytest", line 33 in <module>
The-Compiler commented 3 years ago

Somewhat related: #3804 - also note this goes away with --tb=native as well.

kri-k commented 3 years ago

It happens because of quadratic loop in Traceback.recursionindex(). And comparing magic mocks (they are in f_locals) is very slow (it's several dozen times slower than comparing non-magic mocks).

kri-k commented 3 years ago

Maybe a linear algorithm can be implemented here? Let's say we have an array of frames with same key called f with length equal to n + 1. We want to find recursion start indexes: i and j such that f_i = f_j (in terms of f_locals equality). If this is an infinite recursion, It can be assumed that all subsequent iterations will be exactly the same as they were: f_(i+1) = f_(j+1), f_(i+2) = f_(j+2), .... So, we can start our search from the end of the array: find such k that f_k = f_n. Next, we can try to improve the result: check all pairs f_(k-1) =? f_(n-1), f_(k-2) =? f_(n-2) etc.

UPD This will not work because last frame is not necessarily part of the loop (see: https://github.com/pytest-dev/pytest/pull/8651#issuecomment-846466342).