'ddtrace.contrib.psycopg' has no attribute 'patch'

[mattflo]

Summary of problem

AttributeError: partially initialized module 'ddtrace.contrib.psycopg' has no attribute 'patch' (most likely due to a circular import)

https://github.com/DataDog/dd-trace-py/blob/2.x/ddtrace/_monkey.py#L50C6-L50C13

Which version of dd-trace-py are you using?

I've reproduced this on the latest ddtrace (2.0.2) and the latest psychopg (2.9.9) But it's been happening for a long time with versions on both packages.

Which version of pip are you using?

23.2.1

Which libraries and their versions are you using?

`pip freeze`

aiofiles==23.2.1 aiohttp==3.8.5 aiohttp-retry==2.8.3 aiosignal==1.3.1 alembic==1.11.3 anyio==3.7.1 appdirs==1.4.4 appnope==0.1.3 argon2-cffi==23.1.0 argon2-cffi-bindings==21.2.0 arrow==1.2.3 asgi-correlation-id==4.2.0 astor==0.8.1 asttokens==2.2.1 async-lru==2.0.4 async-timeout==4.0.3 asyncer==0.0.2 attrs==23.1.0 autopep8==2.0.2 backcall==0.2.0 backoff==2.2.1 beautifulsoup4==4.12.2 bidict==0.22.1 bleach==6.0.0 blessed==1.20.0 blinker==1.6.2 boltons==21.0.0 boto3==1.28.34 botocore==1.31.34 bracex==2.4 Brotli==1.0.9 bytecode==0.14.2 cachetools==5.3.1 cattrs==23.1.2 certifi==2023.7.22 cffi==1.15.1 chainlit==0.7.0 chardet==5.2.0 charset-normalizer==3.2.0 chromadb==0.3.26 click==8.1.7 click-option-group==0.5.6 clickhouse-connect==0.6.9 colorama==0.4.6 coloredlogs==15.0.1 comm==0.1.4 ConfigArgParse==1.7 dataclasses-json==0.5.14 ddsketch==2.0.4 ddtrace==1.17.3 debugpy==1.6.7.post1 decorator==5.1.1 defusedxml==0.7.1 Deprecated==1.2.14 dnspython==2.4.2 docopt==0.6.2 duckdb==0.8.1 envier==0.4.0 exceptiongroup==1.1.3 executing==1.2.0 face==22.0.0 Faker==18.13.0 fastapi==0.99.1 fastapi-socketio==0.0.10 fastjsonschema==2.18.0 filelock==3.12.4 filetype==1.2.0 Flask==2.3.3 Flask-BasicAuth==0.2.0 Flask-Cors==4.0.0 flatbuffers==23.5.26 fqdn==1.5.1 frozenlist==1.4.0 fsspec==2023.9.2 gevent==23.7.0 geventhttpclient==2.0.9 git-python==1.0.3 gitdb==4.0.10 GitPython==3.1.32 glom==22.1.0 google-api-core==2.11.1 google-auth==2.22.0 google-auth-oauthlib==1.0.0 google-search-results==2.4.2 googleapis-common-protos==1.60.0 googlemaps==4.10.0 greenlet==2.0.2 grpcio==1.57.0 gspread==5.10.0 h11==0.14.0 hnswlib==0.7.0 httpcore==0.17.3 httptools==0.6.0 httpx==0.24.1 huggingface-hub==0.16.4 humanfriendly==10.0 idna==3.4 importlib-metadata==6.8.0 iniconfig==2.0.0 inquirer==3.1.3 ipykernel==6.25.1 ipython==8.14.0 ipython-genutils==0.2.0 ipywidgets==8.1.0 isoduration==20.11.0 isort==5.12.0 itsdangerous==2.1.2 jedi==0.19.0 Jinja2==3.1.2 jmespath==1.0.1 joblib==1.3.2 json5==0.9.14 jsonpatch==1.33 jsonpointer==2.4 jsonschema==4.19.0 jsonschema-specifications==2023.7.1 jupyter==1.0.0 jupyter-console==6.6.3 jupyter-events==0.7.0 jupyter-lsp==2.2.0 jupyter_client==8.3.0 jupyter_core==5.3.1 jupyter_server==2.7.2 jupyter_server_terminals==0.4.4 jupyterlab==4.0.5 jupyterlab-pygments==0.2.2 jupyterlab-widgets==3.0.8 jupyterlab_server==2.24.0 langchain==0.0.272 langsmith==0.0.26 lark==1.1.7 Lazify==0.4.0 litellm==0.1.817 locust==2.16.1 loguru==0.7.0 lz4==4.3.2 Mako==1.2.4 markdown-it-py==3.0.0 MarkupSafe==2.1.3 marshmallow==3.20.1 matplotlib-inline==0.1.6 mdurl==0.1.2 mistune==3.0.1 mmh3==3.1.0 monotonic==1.6 mpmath==1.3.0 msgpack==1.0.5 multidict==6.0.4 mypy-extensions==1.0.0 nbclient==0.8.0 nbconvert==7.7.4 nbformat==5.9.2 nest-asyncio==1.5.7 nltk==3.8.1 nodeenv==1.8.0 notebook==7.0.2 notebook_shim==0.2.3 numexpr==2.8.5 numpy==1.25.2 oauthlib==3.2.2 onnxruntime==1.15.1 open-interpreter==0.1.7 openai==0.27.9 opentelemetry-api==1.19.0 opentelemetry-exporter-otlp==1.19.0 opentelemetry-exporter-otlp-proto-common==1.19.0 opentelemetry-exporter-otlp-proto-grpc==1.19.0 opentelemetry-exporter-otlp-proto-http==1.19.0 opentelemetry-instrumentation==0.40b0 opentelemetry-proto==1.19.0 opentelemetry-sdk==1.19.0 opentelemetry-semantic-conventions==0.40b0 overrides==7.4.0 packaging==23.1 pandas==2.0.3 pandocfilters==1.5.0 parso==0.8.3 peewee==3.16.3 pexpect==4.8.0 pickleshare==0.7.5 pinecone-client==2.2.2 pinecone-text==0.5.3 platformdirs==3.10.0 pluggy==1.2.0 posthog==3.0.2 prisma==0.10.0 prometheus-client==0.17.1 prompt-toolkit==3.0.39 protobuf==4.24.1 psutil==5.9.5 psycopg==3.1.10 psycopg2-binary==2.9.7 ptyprocess==0.7.0 pulsar-client==3.2.0 pure-eval==0.2.2 pyasn1==0.5.0 pyasn1-modules==0.3.0 pycodestyle==2.11.0 pycparser==2.21 pydantic==1.10.12 Pygments==2.16.1 pygount==1.6.1 PyJWT==2.8.0 pytest==7.4.0 pytest-mock==3.11.1 pytest-notifier==1.0.4 pytest-watch==4.2.0 python-dateutil==2.8.2 python-dotenv==1.0.0 python-editor==1.0.4 python-engineio==4.6.1 python-graphql-client==0.4.3 python-json-logger==2.0.7 python-lsp-jsonrpc==1.0.0 python-multipart==0.0.6 python-socketio==5.8.0 pytz==2023.3 PyYAML==6.0.1 pyzmq==25.1.1 qtconsole==5.4.3 QtPy==2.3.1 readchar==4.0.5 referencing==0.30.2 regex==2023.8.8 requests==2.31.0 requests-oauthlib==1.3.1 rfc3339-validator==0.1.4 rfc3986-validator==0.1.1 rich==13.5.2 roundrobin==0.0.4 rpds-py==0.9.2 rsa==4.9 ruamel.yaml==0.17.33 ruamel.yaml.clib==0.2.8 s3transfer==0.6.2 scipy==1.11.2 semgrep==1.43.0 Send2Trash==1.8.2 six==1.16.0 smmap==5.0.0 sniffio==1.3.0 soupsieve==2.4.1 SQLAlchemy==2.0.20 stack-data==0.6.2 starlette==0.27.0 structlog==23.1.0 sympy==1.12 syncer==2.0.3 tenacity==8.2.3 termcolor==2.3.0 terminado==0.17.1 thefuzz==0.19.0 tiktoken==0.4.0 tinycss2==1.2.1 tokenizers==0.13.3 tokentrim==0.1.10 tomli==2.0.1 tomlkit==0.12.1 tornado==6.3.3 tqdm==4.66.1 traitlets==5.9.0 twilio==8.7.0 typing-inspect==0.9.0 typing_extensions==4.7.1 tzdata==2023.3 ujson==5.8.0 uptrace==1.19.0 uri-template==1.3.0 urllib3==1.26.16 uvicorn==0.23.2 uvloop==0.17.0 watchdog==3.0.0 watchfiles==0.20.0 wcmatch==8.5 wcwidth==0.2.6 webcolors==1.13 webencodings==0.5.1 websocket-client==1.6.2 websockets==11.0.3 Werkzeug==2.3.7 wget==3.2 widgetsnbextension==4.0.8 wrapt==1.15.0 xmltodict==0.13.0 yarl==1.9.2 yaspin==3.0.1 zipp==3.16.2 zope.event==5.0 zope.interface==6.0 zstandard==0.21.0

Can we just remove psychopg from the list of modules for now?

[majorgreys]

@mattflo Where and when are you seeing that exception being raised? Can you include more of the stack trace with the exception?

[mattflo]

@majorgreys when the application starts:

baby-ren-brain-1          | 2023-10-12T16:55:07.308613Z [error    ] failed to import ddtrace module 'ddtrace.contrib.psycopg' when patching on import [ddtrace._monkey] dd.span_id=0 dd.trace_id=0
baby-ren-brain-1          | Traceback (most recent call last):
baby-ren-brain-1          |   File "/usr/local/lib/python3.10/site-packages/ddtrace/_monkey.py", line 160, in on_import
baby-ren-brain-1          |     imported_module = importlib.import_module(path)
baby-ren-brain-1          |   File "/usr/local/lib/python3.10/importlib/__init__.py", line 126, in import_module
baby-ren-brain-1          |     return _bootstrap._gcd_import(name[level:], package, level)
baby-ren-brain-1          |   File "<frozen importlib._bootstrap>", line 1050, in _gcd_import
baby-ren-brain-1          |   File "<frozen importlib._bootstrap>", line 1027, in _find_and_load
baby-ren-brain-1          |   File "<frozen importlib._bootstrap>", line 1006, in _find_and_load_unlocked
baby-ren-brain-1          |   File "<frozen importlib._bootstrap>", line 688, in _load_unlocked
baby-ren-brain-1          |   File "/usr/local/lib/python3.10/site-packages/ddtrace/internal/module.py", line 220, in _exec_module
baby-ren-brain-1          |     self.loader.exec_module(module)
baby-ren-brain-1          |   File "<frozen importlib._bootstrap_external>", line 883, in exec_module
baby-ren-brain-1          |   File "<frozen importlib._bootstrap>", line 241, in _call_with_frames_removed
baby-ren-brain-1          |   File "/usr/local/lib/python3.10/site-packages/ddtrace/contrib/psycopg/__init__.py", line 63, in <module>
baby-ren-brain-1          |     from .patch import patch
baby-ren-brain-1          |   File "<frozen importlib._bootstrap>", line 1027, in _find_and_load
baby-ren-brain-1          |   File "<frozen importlib._bootstrap>", line 1006, in _find_and_load_unlocked
baby-ren-brain-1          |   File "<frozen importlib._bootstrap>", line 688, in _load_unlocked
baby-ren-brain-1          |   File "/usr/local/lib/python3.10/site-packages/ddtrace/internal/module.py", line 220, in _exec_module
baby-ren-brain-1          |     self.loader.exec_module(module)
baby-ren-brain-1          |   File "/usr/local/lib/python3.10/site-packages/ddtrace/contrib/psycopg/patch.py", line 34, in <module>
baby-ren-brain-1          |     psycopg_import = import_module("psycopg")
baby-ren-brain-1          |   File "/usr/local/lib/python3.10/importlib/__init__.py", line 126, in import_module
baby-ren-brain-1          |     return _bootstrap._gcd_import(name[level:], package, level)
baby-ren-brain-1          |   File "<frozen importlib._bootstrap>", line 1050, in _gcd_import
baby-ren-brain-1          |   File "<frozen importlib._bootstrap>", line 1027, in _find_and_load
baby-ren-brain-1          |   File "<frozen importlib._bootstrap>", line 1006, in _find_and_load_unlocked
baby-ren-brain-1          |   File "<frozen importlib._bootstrap>", line 688, in _load_unlocked
baby-ren-brain-1          |   File "/usr/local/lib/python3.10/site-packages/ddtrace/vendor/wrapt/importer.py", line 177, in _exec_module
baby-ren-brain-1          |     notify_module_loaded(module)
baby-ren-brain-1          |   File "/usr/local/lib/python3.10/site-packages/ddtrace/vendor/wrapt/decorators.py", line 470, in _synchronized
baby-ren-brain-1          |     return wrapped(*args, **kwargs)
baby-ren-brain-1          |   File "/usr/local/lib/python3.10/site-packages/ddtrace/vendor/wrapt/importer.py", line 136, in notify_module_loaded
baby-ren-brain-1          |     hook(module)
baby-ren-brain-1          |   File "/usr/local/lib/python3.10/site-packages/ddtrace/_monkey.py", line 168, in on_import
baby-ren-brain-1          |     imported_module.patch()
baby-ren-brain-1          | AttributeError: partially initialized module 'ddtrace.contrib.psycopg' has no attribute 'patch' (most likely due to a circular import)

[gnufede]

So possibly this import of psycopg in ddtrace.contrib.psycopg causes a circular import then? https://github.com/DataDog/dd-trace-py/blob/7ec041bf1f716287f04450f4d012b4c13dde18ed/ddtrace/contrib/psycopg/patch.py#L36

[elch-yan]

I'm having same issue with redis: AttributeError: partially initialized module 'ddtrace.contrib.redis' has no attribute 'patch' (most likely due to a circular import) ddtrace (2.1.1, 2.1.2)

[emmettbutler]

@elch-yan see https://github.com/DataDog/dd-trace-py/issues/7783 for the redis issue.