search

DataDog / dd-trace-py

Datadog Python APM Client
https://ddtrace.readthedocs.io/
Other
550 stars 413 forks source link

Circular import error between redis integrations #7783

Closed beepboop271 closed 11 months ago

beepboop271 commented 11 months ago

Summary of problem

aredis, aioredis, and yaaredis integrations import from the redis integration.

If for example aredis and redis are both installed, a circular import occurs when importing aredis after patching.

This same error has been an issue before: https://github.com/DataDog/dd-trace-py/issues/5601#issuecomment-1516265155

and fixed before: https://github.com/DataDog/dd-trace-py/pull/5608, by moving the utils being imported from the redis integration into a common trace_utils_redis file.

However a cross-integration import was added a few months ago, breaking it again: https://github.com/DataDog/dd-trace-py/blob/67c5ec7a4c397199014823f649a810c708a6113d/ddtrace/contrib/aredis/patch.py#L14-L16

https://github.com/DataDog/dd-trace-py/blob/v2.3.1/ddtrace/contrib/aredis/patch.py#L14-L16

Which version of dd-trace-py are you using?

2.3.1

Which version of pip are you using?

pip 23.3.1 python 3.9.9

Which libraries and their versions are you using?

pip freeze:

aredis==1.1.8
async-timeout==4.0.3
attrs==23.1.0
bytecode==0.15.1
cattrs==23.2.2
ddsketch==2.0.4
ddtrace==2.3.1
Deprecated==1.2.14
envier==0.4.0
exceptiongroup==1.2.0
importlib-metadata==6.8.0
opentelemetry-api==1.21.0
protobuf==4.25.1
redis==5.0.1
six==1.16.0
typing_extensions==4.8.0
wrapt==1.16.0
xmltodict==0.13.0
zipp==3.17.0

How can we reproduce your problem?

Minimal reproduction:

  1. fresh venv
  2. pip install ddtrace aredis redis
  3. run:
from ddtrace import patch_all
patch_all()
import aredis

What is the result that you get?

failed to import ddtrace module 'ddtrace.contrib.aredis' when patching on import
Traceback (most recent call last):
  File "/workspace/test/.venv/lib/python3.9/site-packages/ddtrace/_monkey.py", line 165, in on_import
    imported_module = importlib.import_module(path)
  File "/home/coder/.pyenv/versions/3.9.9/lib/python3.9/importlib/__init__.py", line 127, in import_module
    return _bootstrap._gcd_import(name[level:], package, level)
  File "<frozen importlib._bootstrap>", line 1030, in _gcd_import
  File "<frozen importlib._bootstrap>", line 1007, in _find_and_load
  File "<frozen importlib._bootstrap>", line 986, in _find_and_load_unlocked
  File "<frozen importlib._bootstrap>", line 680, in _load_unlocked
  File "/workspace/test/.venv/lib/python3.9/site-packages/ddtrace/internal/module.py", line 211, in _exec_module
    self.loader.exec_module(module)
  File "<frozen importlib._bootstrap_external>", line 850, in exec_module
  File "<frozen importlib._bootstrap>", line 228, in _call_with_frames_removed
  File "/workspace/test/.venv/lib/python3.9/site-packages/ddtrace/contrib/aredis/__init__.py", line 76, in <module>
    from .patch import get_version
  File "<frozen importlib._bootstrap>", line 1007, in _find_and_load
  File "<frozen importlib._bootstrap>", line 986, in _find_and_load_unlocked
  File "<frozen importlib._bootstrap>", line 680, in _load_unlocked
  File "/workspace/test/.venv/lib/python3.9/site-packages/ddtrace/internal/module.py", line 211, in _exec_module
    self.loader.exec_module(module)
  File "/workspace/test/.venv/lib/python3.9/site-packages/ddtrace/contrib/aredis/patch.py", line 14, in <module>
    from ..redis.asyncio_patch import _run_redis_command_async
  File "<frozen importlib._bootstrap>", line 1007, in _find_and_load
  File "<frozen importlib._bootstrap>", line 986, in _find_and_load_unlocked
  File "<frozen importlib._bootstrap>", line 680, in _load_unlocked
  File "/workspace/test/.venv/lib/python3.9/site-packages/ddtrace/internal/module.py", line 211, in _exec_module
    self.loader.exec_module(module)
  File "/workspace/test/.venv/lib/python3.9/site-packages/ddtrace/contrib/redis/__init__.py", line 75, in <module>
    with require_modules(required_modules) as missing_modules:
  File "/workspace/test/.venv/lib/python3.9/site-packages/ddtrace/internal/utils/importlib.py", line 20, in __init__
    import_module(module)
  File "/home/coder/.pyenv/versions/3.9.9/lib/python3.9/importlib/__init__.py", line 127, in import_module
    return _bootstrap._gcd_import(name[level:], package, level)
  File "<frozen importlib._bootstrap>", line 1030, in _gcd_import
  File "<frozen importlib._bootstrap>", line 1007, in _find_and_load
  File "<frozen importlib._bootstrap>", line 986, in _find_and_load_unlocked
  File "<frozen importlib._bootstrap>", line 680, in _load_unlocked
  File "/workspace/test/.venv/lib/python3.9/site-packages/ddtrace/vendor/wrapt/importer.py", line 177, in _exec_module
    notify_module_loaded(module)
  File "/workspace/test/.venv/lib/python3.9/site-packages/ddtrace/vendor/wrapt/decorators.py", line 470, in _synchronized
    return wrapped(*args, **kwargs)
  File "/workspace/test/.venv/lib/python3.9/site-packages/ddtrace/vendor/wrapt/importer.py", line 136, in notify_module_loaded
    hook(module)
  File "/workspace/test/.venv/lib/python3.9/site-packages/ddtrace/_monkey.py", line 176, in on_import
    imported_module.patch()
AttributeError: partially initialized module 'ddtrace.contrib.redis' has no attribute 'patch' (most likely due to a circular import)

Notice the frame about halfway through the trace:

  File "/workspace/test/.venv/lib/python3.9/site-packages/ddtrace/contrib/aredis/patch.py", line 14, in <module>
    from ..redis.asyncio_patch import _run_redis_command_async

What is the result that you expected?

No error thrown

emmettbutler commented 11 months ago

I'm going to close this as a duplicate of https://github.com/DataDog/dd-trace-py/issues/7241

emmettbutler commented 11 months ago

On second thought, there's an important difference between the two.