search

ets-labs / python-dependency-injector

Dependency injection framework for Python
https://python-dependency-injector.ets-labs.org/
BSD 3-Clause "New" or "Revised" License
3.89k stars 304 forks source link

How to debug RecursionError? #624

Open chbndrhnns opened 2 years ago

chbndrhnns commented 2 years ago

I have a nested structure of containers and I wanted to perform some refactoring by adding the new container structure first and then gradually removing the current structure.

However, DI decides to throw a RecursionError and I'd like to debug that. Can anyone help how to do it? Here is the stacktrace so far. I was not yet able to narrow the example down to a minimal working example, unfortunately.

Traceback (most recent call last):
  File "src/dependency_injector/providers.pyx", line 4920, in dependency_injector.providers.deepcopy (src/dependency_injector/providers.c:97879)
  File "/Users/cleancoder/.pyenv/versions/3.9.12/lib/python3.9/copy.py", line 146, in deepcopy
    y = copier(x, memo)
  File "/Users/cleancoder/.pyenv/versions/3.9.12/lib/python3.9/copy.py", line 210, in _deepcopy_tuple
    y = [deepcopy(a, memo) for a in x]
  File "/Users/cleancoder/.pyenv/versions/3.9.12/lib/python3.9/copy.py", line 210, in <listcomp>
    y = [deepcopy(a, memo) for a in x]
  File "/Users/cleancoder/.pyenv/versions/3.9.12/lib/python3.9/copy.py", line 153, in deepcopy
    y = copier(memo)
  File "src/dependency_injector/providers.pyx", line 798, in dependency_injector.providers.Dependency.__deepcopy__ (src/dependency_injector/providers.c:18613)
  File "src/dependency_injector/providers.pyx", line 942, in dependency_injector.providers.Dependency._copy_parent (src/dependency_injector/providers.c:21090)
  File "src/dependency_injector/providers.pyx", line 5043, in dependency_injector.providers._copy_parent (src/dependency_injector/providers.c:100267)
  File "src/dependency_injector/providers.pyx", line 4920, in dependency_injector.providers.deepcopy (src/dependency_injector/providers.c:97879)
  File "/Users/cleancoder/.pyenv/versions/3.9.12/lib/python3.9/copy.py", line 153, in deepcopy
    y = copier(memo)
  File "src/dependency_injector/providers.pyx", line 1046, in dependency_injector.providers.DependenciesContainer.__deepcopy__ (src/dependency_injector/providers.c:22854)
  File "src/dependency_injector/providers.pyx", line 4920, in dependency_injector.providers.deepcopy (src/dependency_injector/providers.c:97879)
  File "/Users/cleancoder/.pyenv/versions/3.9.12/lib/python3.9/copy.py", line 146, in deepcopy
    y = copier(x, memo)
  File "/Users/cleancoder/.pyenv/versions/3.9.12/lib/python3.9/copy.py", line 230, in _deepcopy_dict
    y[deepcopy(key, memo)] = deepcopy(value, memo)
  File "/Users/cleancoder/.pyenv/versions/3.9.12/lib/python3.9/copy.py", line 153, in deepcopy
    y = copier(memo)
  File "src/dependency_injector/providers.pyx", line 790, in dependency_injector.providers.Dependency.__deepcopy__ (src/dependency_injector/providers.c:18434)
RecursionError: maximum recursion depth exceeded while calling a Python object
Traceback (most recent call last):
  File "src/dependency_injector/providers.pyx", line 4920, in dependency_injector.providers.deepcopy (src/dependency_injector/providers.c:97879)
  File "/Users/cleancoder/.pyenv/versions/3.9.12/lib/python3.9/copy.py", line 146, in deepcopy
    y = copier(x, memo)
  File "/Users/cleancoder/.pyenv/versions/3.9.12/lib/python3.9/copy.py", line 210, in _deepcopy_tuple
    y = [deepcopy(a, memo) for a in x]
  File "/Users/cleancoder/.pyenv/versions/3.9.12/lib/python3.9/copy.py", line 210, in <listcomp>
    y = [deepcopy(a, memo) for a in x]
  File "/Users/cleancoder/.pyenv/versions/3.9.12/lib/python3.9/copy.py", line 153, in deepcopy
    y = copier(memo)
  File "src/dependency_injector/providers.pyx", line 798, in dependency_injector.providers.Dependency.__deepcopy__ (src/dependency_injector/providers.c:18613)
  File "src/dependency_injector/providers.pyx", line 942, in dependency_injector.providers.Dependency._copy_parent (src/dependency_injector/providers.c:21090)
  File "src/dependency_injector/providers.pyx", line 5043, in dependency_injector.providers._copy_parent (src/dependency_injector/providers.c:100267)
  File "src/dependency_injector/providers.pyx", line 4920, in dependency_injector.providers.deepcopy (src/dependency_injector/providers.c:97879)
  File "/Users/cleancoder/.pyenv/versions/3.9.12/lib/python3.9/copy.py", line 137, in deepcopy
    d = id(x)
RecursionError: maximum recursion depth exceeded while calling a Python object
Traceback (most recent call last):
  File "src/dependency_injector/providers.pyx", line 4920, in dependency_injector.providers.deepcopy (src/dependency_injector/providers.c:97879)
  File "/Users/cleancoder/.pyenv/versions/3.9.12/lib/python3.9/copy.py", line 146, in deepcopy
    y = copier(x, memo)
  File "/Users/cleancoder/.pyenv/versions/3.9.12/lib/python3.9/copy.py", line 210, in _deepcopy_tuple
    y = [deepcopy(a, memo) for a in x]
  File "/Users/cleancoder/.pyenv/versions/3.9.12/lib/python3.9/copy.py", line 210, in <listcomp>
    y = [deepcopy(a, memo) for a in x]
  File "/Users/cleancoder/.pyenv/versions/3.9.12/lib/python3.9/copy.py", line 153, in deepcopy
    y = copier(memo)
  File "src/dependency_injector/providers.pyx", line 2832, in dependency_injector.providers.BaseSingleton.__deepcopy__ (src/dependency_injector/providers.c:55991)
  File "src/dependency_injector/providers.pyx", line 5051, in dependency_injector.providers._memorized_duplicate (src/dependency_injector/providers.c:100488)
  File "src/dependency_injector/providers.pyx", line 3034, in dependency_injector.providers.Singleton.__init__ (src/dependency_injector/providers.c:59024)
  File "src/dependency_injector/providers.pyx", line 2812, in dependency_injector.providers.BaseSingleton.__init__ (src/dependency_injector/providers.c:55670)
  File "src/dependency_injector/providers.pyx", line 2517, in dependency_injector.providers.Factory.__init__ (src/dependency_injector/providers.c:50156)
RecursionError: maximum recursion depth exceeded while calling a Python object
Traceback (most recent call last):
  File "src/dependency_injector/providers.pyx", line 4920, in dependency_injector.providers.deepcopy (src/dependency_injector/providers.c:97879)
  File "/Users/cleancoder/.pyenv/versions/3.9.12/lib/python3.9/copy.py", line 146, in deepcopy
    y = copier(x, memo)
  File "/Users/cleancoder/.pyenv/versions/3.9.12/lib/python3.9/copy.py", line 210, in _deepcopy_tuple
    y = [deepcopy(a, memo) for a in x]
  File "/Users/cleancoder/.pyenv/versions/3.9.12/lib/python3.9/copy.py", line 210, in <listcomp>
    y = [deepcopy(a, memo) for a in x]
  File "/Users/cleancoder/.pyenv/versions/3.9.12/lib/python3.9/copy.py", line 153, in deepcopy
    y = copier(memo)
  File "src/dependency_injector/providers.pyx", line 2832, in dependency_injector.providers.BaseSingleton.__deepcopy__ (src/dependency_injector/providers.c:55991)
  File "src/dependency_injector/providers.pyx", line 5051, in dependency_injector.providers._memorized_duplicate (src/dependency_injector/providers.c:100488)
  File "src/dependency_injector/providers.pyx", line 3264, in dependency_injector.providers.ContextLocalSingleton.__init__ (src/dependency_injector/providers.c:64048)
  File "src/dependency_injector/providers.pyx", line 2812, in dependency_injector.providers.BaseSingleton.__init__ (src/dependency_injector/providers.c:55670)
  File "src/dependency_injector/providers.pyx", line 2517, in dependency_injector.providers.Factory.__init__ (src/dependency_injector/providers.c:50156)
  File "src/dependency_injector/providers.pyx", line 1195, in dependency_injector.providers.Callable.__init__ (src/dependency_injector/providers.c:25701)
  File "src/dependency_injector/providers.pyx", line 1234, in dependency_injector.providers.Callable.set_provides (src/dependency_injector/providers.c:26287)
RecursionError: maximum recursion depth exceeded while calling a Python object
Traceback (most recent call last):
  File "src/dependency_injector/providers.pyx", line 4920, in dependency_injector.providers.deepcopy (src/dependency_injector/providers.c:97879)
  File "/Users/cleancoder/.pyenv/versions/3.9.12/lib/python3.9/copy.py", line 146, in deepcopy
    y = copier(x, memo)
  File "/Users/cleancoder/.pyenv/versions/3.9.12/lib/python3.9/copy.py", line 210, in _deepcopy_tuple
    y = [deepcopy(a, memo) for a in x]
  File "/Users/cleancoder/.pyenv/versions/3.9.12/lib/python3.9/copy.py", line 210, in <listcomp>
    y = [deepcopy(a, memo) for a in x]
  File "/Users/cleancoder/.pyenv/versions/3.9.12/lib/python3.9/copy.py", line 153, in deepcopy
    y = copier(memo)
  File "src/dependency_injector/providers.pyx", line 798, in dependency_injector.providers.Dependency.__deepcopy__ (src/dependency_injector/providers.c:18613)
  File "src/dependency_injector/providers.pyx", line 942, in dependency_injector.providers.Dependency._copy_parent (src/dependency_injector/providers.c:21090)
  File "src/dependency_injector/providers.pyx", line 5043, in dependency_injector.providers._copy_parent (src/dependency_injector/providers.c:100267)
  File "src/dependency_injector/providers.pyx", line 4920, in dependency_injector.providers.deepcopy (src/dependency_injector/providers.c:97879)
  File "/Users/cleancoder/.pyenv/versions/3.9.12/lib/python3.9/copy.py", line 153, in deepcopy
    y = copier(memo)
  File "src/dependency_injector/providers.pyx", line 1046, in dependency_injector.providers.DependenciesContainer.__deepcopy__ (src/dependency_injector/providers.c:22854)
  File "src/dependency_injector/providers.pyx", line 4920, in dependency_injector.providers.deepcopy (src/dependency_injector/providers.c:97879)
  File "/Users/cleancoder/.pyenv/versions/3.9.12/lib/python3.9/copy.py", line 146, in deepcopy
    y = copier(x, memo)
  File "/Users/cleancoder/.pyenv/versions/3.9.12/lib/python3.9/copy.py", line 230, in _deepcopy_dict
    y[deepcopy(key, memo)] = deepcopy(value, memo)
  File "/Users/cleancoder/.pyenv/versions/3.9.12/lib/python3.9/copy.py", line 153, in deepcopy
    y = copier(memo)
  File "src/dependency_injector/providers.pyx", line 794, in dependency_injector.providers.Dependency.__deepcopy__ (src/dependency_injector/providers.c:18495)
  File "src/dependency_injector/providers.pyx", line 5051, in dependency_injector.providers._memorized_duplicate (src/dependency_injector/providers.c:100488)
  File "src/dependency_injector/providers.pyx", line 779, in dependency_injector.providers.Dependency.__init__ (src/dependency_injector/providers.c:18269)
  File "src/dependency_injector/providers.pyx", line 875, in dependency_injector.providers.Dependency.set_instance_of (src/dependency_injector/providers.c:20047)
RecursionError: maximum recursion depth exceeded in __instancecheck__
Traceback (most recent call last):
  File "src/dependency_injector/providers.pyx", line 4920, in dependency_injector.providers.deepcopy (src/dependency_injector/providers.c:97879)
  File "/Users/cleancoder/.pyenv/versions/3.9.12/lib/python3.9/copy.py", line 146, in deepcopy
    y = copier(x, memo)
  File "/Users/cleancoder/.pyenv/versions/3.9.12/lib/python3.9/copy.py", line 210, in _deepcopy_tuple
    y = [deepcopy(a, memo) for a in x]
  File "/Users/cleancoder/.pyenv/versions/3.9.12/lib/python3.9/copy.py", line 210, in <listcomp>
    y = [deepcopy(a, memo) for a in x]
  File "/Users/cleancoder/.pyenv/versions/3.9.12/lib/python3.9/copy.py", line 153, in deepcopy
    y = copier(memo)
  File "src/dependency_injector/providers.pyx", line 2832, in dependency_injector.providers.BaseSingleton.__deepcopy__ (src/dependency_injector/providers.c:55991)
  File "src/dependency_injector/providers.pyx", line 5051, in dependency_injector.providers._memorized_duplicate (src/dependency_injector/providers.c:100488)
  File "src/dependency_injector/providers.pyx", line 3264, in dependency_injector.providers.ContextLocalSingleton.__init__ (src/dependency_injector/providers.c:64048)
  File "src/dependency_injector/providers.pyx", line 2812, in dependency_injector.providers.BaseSingleton.__init__ (src/dependency_injector/providers.c:55670)
  File "src/dependency_injector/providers.pyx", line 2517, in dependency_injector.providers.Factory.__init__ (src/dependency_injector/providers.c:50156)
  File "src/dependency_injector/providers.pyx", line 1195, in dependency_injector.providers.Callable.__init__ (src/dependency_injector/providers.c:25701)
  File "src/dependency_injector/providers.pyx", line 1234, in dependency_injector.providers.Callable.set_provides (src/dependency_injector/providers.c:26287)
RecursionError: maximum recursion depth exceeded while calling a Python object
Traceback (most recent call last):
  File "src/dependency_injector/providers.pyx", line 4920, in dependency_injector.providers.deepcopy (src/dependency_injector/providers.c:97879)
  File "/Users/cleancoder/.pyenv/versions/3.9.12/lib/python3.9/copy.py", line 146, in deepcopy
    y = copier(x, memo)
  File "/Users/cleancoder/.pyenv/versions/3.9.12/lib/python3.9/copy.py", line 210, in _deepcopy_tuple
    y = [deepcopy(a, memo) for a in x]
  File "/Users/cleancoder/.pyenv/versions/3.9.12/lib/python3.9/copy.py", line 210, in <listcomp>
    y = [deepcopy(a, memo) for a in x]
  File "/Users/cleancoder/.pyenv/versions/3.9.12/lib/python3.9/copy.py", line 153, in deepcopy
    y = copier(memo)
  File "src/dependency_injector/providers.pyx", line 2832, in dependency_injector.providers.BaseSingleton.__deepcopy__ (src/dependency_injector/providers.c:55991)
  File "src/dependency_injector/providers.pyx", line 5051, in dependency_injector.providers._memorized_duplicate (src/dependency_injector/providers.c:100488)
  File "src/dependency_injector/providers.pyx", line 3264, in dependency_injector.providers.ContextLocalSingleton.__init__ (src/dependency_injector/providers.c:64048)
  File "src/dependency_injector/providers.pyx", line 2812, in dependency_injector.providers.BaseSingleton.__init__ (src/dependency_injector/providers.c:55670)
  File "src/dependency_injector/providers.pyx", line 2517, in dependency_injector.providers.Factory.__init__ (src/dependency_injector/providers.c:50156)
  File "src/dependency_injector/providers.pyx", line 1195, in dependency_injector.providers.Callable.__init__ (src/dependency_injector/providers.c:25701)
  File "src/dependency_injector/providers.pyx", line 1234, in dependency_injector.providers.Callable.set_provides (src/dependency_injector/providers.c:26287)
RecursionError: maximum recursion depth exceeded while calling a Python object
Traceback (most recent call last):
  File "src/dependency_injector/providers.pyx", line 4920, in dependency_injector.providers.deepcopy (src/dependency_injector/providers.c:97879)
  File "/Users/cleancoder/.pyenv/versions/3.9.12/lib/python3.9/copy.py", line 146, in deepcopy
    y = copier(x, memo)
  File "/Users/cleancoder/.pyenv/versions/3.9.12/lib/python3.9/copy.py", line 210, in _deepcopy_tuple
    y = [deepcopy(a, memo) for a in x]
  File "/Users/cleancoder/.pyenv/versions/3.9.12/lib/python3.9/copy.py", line 210, in <listcomp>
    y = [deepcopy(a, memo) for a in x]
  File "/Users/cleancoder/.pyenv/versions/3.9.12/lib/python3.9/copy.py", line 153, in deepcopy
    y = copier(memo)
  File "src/dependency_injector/providers.pyx", line 1047, in dependency_injector.providers.DependenciesContainer.__deepcopy__ (src/dependency_injector/providers.c:22913)
  File "src/dependency_injector/providers.pyx", line 1155, in dependency_injector.providers.DependenciesContainer._copy_parent (src/dependency_injector/providers.c:24847)
  File "src/dependency_injector/providers.pyx", line 5043, in dependency_injector.providers._copy_parent (src/dependency_injector/providers.c:100267)
  File "src/dependency_injector/providers.pyx", line 4920, in dependency_injector.providers.deepcopy (src/dependency_injector/providers.c:97879)
  File "/Users/cleancoder/.pyenv/versions/3.9.12/lib/python3.9/copy.py", line 153, in deepcopy
    y = copier(memo)
  File "src/dependency_injector/containers.pyx", line 118, in dependency_injector.containers.DynamicContainer.__deepcopy__ (src/dependency_injector/containers.c:5366)
  File "src/dependency_injector/providers.pyx", line 4913, in dependency_injector.providers.deepcopy (src/dependency_injector/providers.c:98022)
  File "src/dependency_injector/providers.pyx", line 4920, in dependency_injector.providers.deepcopy (src/dependency_injector/providers.c:97879)
  File "/Users/cleancoder/.pyenv/versions/3.9.12/lib/python3.9/copy.py", line 153, in deepcopy
    y = copier(memo)
  File "src/dependency_injector/providers.pyx", line 537, in dependency_injector.providers.Self.__deepcopy__ (src/dependency_injector/providers.c:13488)
  File "src/dependency_injector/providers.pyx", line 4920, in dependency_injector.providers.deepcopy (src/dependency_injector/providers.c:97879)
  File "/Users/cleancoder/.pyenv/versions/3.9.12/lib/python3.9/copy.py", line 138, in deepcopy
    y = memo.get(d, _nil)
RecursionError: maximum recursion depth exceeded in comparison
Traceback (most recent call last):
  File "src/dependency_injector/providers.pyx", line 4920, in dependency_injector.providers.deepcopy (src/dependency_injector/providers.c:97879)
  File "/Users/cleancoder/.pyenv/versions/3.9.12/lib/python3.9/copy.py", line 146, in deepcopy
    y = copier(x, memo)
  File "/Users/cleancoder/.pyenv/versions/3.9.12/lib/python3.9/copy.py", line 210, in _deepcopy_tuple
    y = [deepcopy(a, memo) for a in x]
  File "/Users/cleancoder/.pyenv/versions/3.9.12/lib/python3.9/copy.py", line 210, in <listcomp>
    y = [deepcopy(a, memo) for a in x]
  File "/Users/cleancoder/.pyenv/versions/3.9.12/lib/python3.9/copy.py", line 153, in deepcopy
    y = copier(memo)
  File "src/dependency_injector/providers.pyx", line 4021, in dependency_injector.providers.Container.__deepcopy__ (src/dependency_injector/providers.c:77945)
  File "src/dependency_injector/providers.pyx", line 4920, in dependency_injector.providers.deepcopy (src/dependency_injector/providers.c:97879)
  File "/Users/cleancoder/.pyenv/versions/3.9.12/lib/python3.9/copy.py", line 153, in deepcopy
    y = copier(memo)
  File "src/dependency_injector/containers.pyx", line 118, in dependency_injector.containers.DynamicContainer.__deepcopy__ (src/dependency_injector/containers.c:5366)
  File "src/dependency_injector/providers.pyx", line 4913, in dependency_injector.providers.deepcopy (src/dependency_injector/providers.c:98022)
  File "src/dependency_injector/providers.pyx", line 4920, in dependency_injector.providers.deepcopy (src/dependency_injector/providers.c:97879)
  File "/Users/cleancoder/.pyenv/versions/3.9.12/lib/python3.9/copy.py", line 153, in deepcopy
    y = copier(memo)
  File "src/dependency_injector/providers.pyx", line 537, in dependency_injector.providers.Self.__deepcopy__ (src/dependency_injector/providers.c:13488)
  File "src/dependency_injector/providers.pyx", line 4920, in dependency_injector.providers.deepcopy (src/dependency_injector/providers.c:97879)
  File "/Users/cleancoder/.pyenv/versions/3.9.12/lib/python3.9/copy.py", line 138, in deepcopy
    y = memo.get(d, _nil)
RecursionError: maximum recursion depth exceeded in comparison

test setup failed
@pytest.fixture
    def web_container():
        from app.core.cloudkitty.container import CloudKittyCore
        from app.core.report.container import ReportCore

        cores = Cores(report=ReportCore, cloudkitty=CloudKittyCore)
>       return WebContainer(cores=cores)
dademon333 commented 1 year ago

Have same issue. Any updates or solutions?

iliamir commented 1 year ago

I have the same issue, temporarily fixed this by increasing the recursion limit for container initialisation. Which is not a perfect solution, but it works.

sys.setrecursionlimit(2000)
container = Container()
sys.setrecursionlimit(1000)