search

DataDog / dd-trace-py

Datadog Python APM Client
https://ddtrace.readthedocs.io/
Other
519 stars 407 forks source link

RuntimeWarnings in IPython when asyncio integration is active #8907

Open iamlikeme opened 5 months ago

iamlikeme commented 5 months ago

Summary of problem

IPython produces RuntimeWarning on each tab-completion and enter when ddtrace's asyncio integration is active.

Which version of dd-trace-py are you using?

2.8.0

Which version of pip are you using?

24.0

Which libraries and their versions are you using?

ipython==8.23.0
ddtrace==2.8.0
`pip freeze` asttokens==2.4.1 attrs==23.2.0 bytecode==0.15.1 cattrs==23.2.3 ddsketch==3.0.1 ddtrace==2.8.0 decorator==5.1.1 Deprecated==1.2.14 envier==0.5.1 executing==2.0.1 importlib-metadata==7.0.0 ipython==8.23.0 jedi==0.19.1 matplotlib-inline==0.1.6 opentelemetry-api==1.24.0 parso==0.8.4 pexpect==4.9.0 prompt-toolkit==3.0.43 protobuf==5.26.1 ptyprocess==0.7.0 pure-eval==0.2.2 Pygments==2.17.2 setuptools==69.2.0 six==1.16.0 sqlparse==0.4.4 stack-data==0.6.3 traitlets==5.14.2 typing_extensions==4.11.0 wcwidth==0.2.13 wrapt==1.16.0 xmltodict==0.13.0 zipp==3.18.1

How can we reproduce your problem?

In an IPython REPL:

In [1]: import ddtrace

In [2]: ddtrace.patch_all()

In [3]: # press enter
/home/piotr/repos/ddtrace-bug/.venv/lib/python3.12/site-packages/prompt_toolkit/application/application.py:1207: RuntimeWarning: coroutine 'Application.run_async.<locals>._run_async.<locals>.auto_flush_input' was never awaited
  await asyncio.wait(
RuntimeWarning: Enable tracemalloc to get the object allocation traceback

/home/piotr/repos/ddtrace-bug/.venv/lib/python3.12/site-packages/prompt_toolkit/key_binding/key_processor.py:413: RuntimeWarning: coroutine 'KeyProcessor._start_timeout.<locals>.wait' was never awaited
  self._flush_wait_task = app.create_background_task(wait())
RuntimeWarning: Enable tracemalloc to get the object allocation traceback

In [3]:

The problem disappears when asyncio integration is deactivated:

In [1]: import ddtrace

In [2]: ddtrace.patch_all(asyncio=False)

In [3]: # press enter

In [3]:

What is the result that you get?

I see RuntimeWarnings on every enter and tab-completion in IPython.

What is the result that you expected?

I expected to see no RuntimeWarnings :slightly_smiling_face:

emmettbutler commented 5 months ago

Interesting! Thanks for the report, @iamlikeme. We'll check it out.

darkwind-noveo commented 3 months ago

I encountered same issue when navigating with tmate in Ipython shell console

In [1]: ls
/usr/local/lib/python3.11/site-packages/prompt_toolkit/application/application.py:1207: RuntimeWarning: coroutine 'Application.run_async.<locals>._run_async.<locals>.auto_flush_input' was never awaited
  await asyncio.wait(
RuntimeWarning: Enable tracemalloc to get the object allocation traceback
CONTRIBUTING.md  Pipfile.lock  certificates/    scripts/    tests/
Dockerfile       README.md     docker/        locale/       pylintrc    setup.cfg   uwsgi-json.ini
Pipfile            manage.py*    pytest.ini  templates/  uwsgi.ini

/usr/local/lib/python3.11/site-packages/prompt_toolkit/key_binding/key_processor.py:413: RuntimeWarning: coroutine 'KeyProcessor._start_timeout.<locals>.wait' was never awaited
  self._flush_wait_task = app.create_background_task(wait())
RuntimeWarning: Enable tracemalloc to get the object allocation traceback

When i downgraded ddtrace from 2.8.2 to 2.7.2 it resolved my issues. (2.7.10 is good too, 2.8.0 breaks)

I tried turning on python tracemalloc stuff via env var, then written error changed itself to

In [1]: 
/usr/local/lib/python3.11/site-packages/prompt_toolkit/application/application.py:1207: RuntimeWarning: coroutine 'Application.run_async.<locals>._run_async.<locals>.auto_flush_input' was never awaited
  await asyncio.wait(
Object allocated at (most recent call last):
  File "/usr/local/lib/python3.11/site-packages/prompt_toolkit/application/application.py", lineno 704
    flush_task = self.create_background_task(auto_flush_input())

/usr/local/lib/python3.11/site-packages/prompt_toolkit/key_binding/key_processor.py:413: RuntimeWarning: coroutine 'KeyProcessor._start_timeout.<locals>.wait' was never awaited
  self._flush_wait_task = app.create_background_task(wait())
Object allocated at (most recent call last):
  File "/usr/local/lib/python3.11/site-packages/prompt_toolkit/key_binding/key_processor.py", lineno 413
    self._flush_wait_task = app.create_background_task(wait())
My pip freeze aiohttp==3.9.5 aioresponses==0.7.6 aiosignal==1.3.1 aiostream==0.5.2 amqp==5.2.0 annotated-types==0.6.0 appdirs==1.4.4 asgiref==3.8.1 astroid==3.1.0 asttokens==2.4.1 async-timeout==4.0.3 asyncpg==0.29.0 attrs==23.2.0 Babel==2.14.0 bcrypt==4.1.2 beautifulsoup4==4.12.3 billiard==4.2.0 boto3==1.28.80 botocore==1.31.80 braintree==3.35.0 Brotli==1.1.0 bytecode==0.15.1 cached-property==1.5.2 cartomancer==0.1.4 cattrs==23.2.3 celery==5.3.1 certifi==2024.2.2 cffi==1.16.0 charset-normalizer==3.3.2 click==8.1.7 click-didyoumean==0.3.1 click-plugins==1.1.1 click-repl==0.3.0 colorama==0.4.6 coverage==6.5.0 coveralls==3.3.1 cron-descriptor==1.4.3 cryptography==42.0.5 cssselect2==0.7.0 cssutils==2.10.2 ddsketch==3.0.1 ddtrace==2.8.2 debugpy==1.8.1 decorator==5.1.1 defusedxml==0.7.1 Deprecated==1.2.14 dill==0.3.8 distlib==0.3.8 Django==4.1.13 django-add-default-value==0.10.0 django-appconf==1.0.6 django-cassette==0.8.1 django-celery-beat==2.5.0 django-chunkator==2.0.0 django-cors-headers==4.3.1 django-countries==7.6.1 django-crispy-forms==2.0 django-datadog-logger==0.7.1 django-extensions==3.2.3 django-filter==22.1 django-fsm==3.0.0 django-fsm-log==3.1.0 django-localflavor==4.0 django-model-utils==4.5.0 django-ratelimit==4.1.0 django-redis==5.4.0 django-relativedelta==2.0.0 django-request-id==1.0.0 django-ses==4.0.0 django-storages==1.14.2 django-timezone-field==6.1.0 django-tqdm==1.3.1 djangorestframework==3.14.0 djangorestframework-filters==1.0.0.dev2 docker==7.0.0 docopt==0.6.2 drf-flex-fields==1.0.2 elastic-apm==2.1.1 elasticsearch==5.5.3 elasticsearch-dsl==5.4.0 elementpath==4.4.0 envier==0.5.1 et-xmlfile==1.1.0 execnet==2.1.1 executing==2.0.1 factory-boy==3.2.1 Faker==24.11.0 filelock==3.14.0 flake8==7.0.0 flake8-tidy-imports==4.10.0 flowey==1.4.5 fonttools==4.51.0 freezegun==1.4.0 frozenlist==1.4.1 getconf==1.11.1 googleapis-common-protos==1.63.0 grpcio==1.62.2 grpcio-tools==1.62.2 html2text==2024.2.26 html5lib==1.1 idna==3.7 importlib-metadata==7.0.0 iniconfig==2.0.0 ipaddress==1.0.23 ipython==8.23.0 isodate==0.6.1 isort==5.13.2 jdcal==1.4.1 jedi==0.19.1 jmespath==1.0.1 JSON-log-formatter==1.0 jsonfield==3.1.0 junitparser==3.1.2 jwcrypto==0.8 kombu==5.3.1 lark==1.1.9 lxml==5.2.1 Markdown==3.6 MarkupSafe==2.1.5 matplotlib-inline==0.1.7 mccabe==0.7.0 mistql==0.4.11 multidict==6.0.5 oauthlib==3.2.2 openpyxl==3.0.6 opentelemetry-api==1.24.0 opentelemetry-exporter-otlp==1.24.0 opentelemetry-exporter-otlp-proto-common==1.24.0 opentelemetry-exporter-otlp-proto-grpc==1.24.0 opentelemetry-exporter-otlp-proto-http==1.24.0 opentelemetry-instrumentation==0.45b0 opentelemetry-instrumentation-asyncpg==0.45b0 opentelemetry-instrumentation-boto==0.45b0 opentelemetry-instrumentation-boto3sqs==0.45b0 opentelemetry-instrumentation-celery==0.45b0 opentelemetry-instrumentation-dbapi==0.45b0 opentelemetry-instrumentation-django==0.45b0 opentelemetry-instrumentation-elasticsearch==0.45b0 opentelemetry-instrumentation-psycopg2==0.45b0 opentelemetry-instrumentation-redis==0.45b0 opentelemetry-instrumentation-requests==0.45b0 opentelemetry-instrumentation-wsgi==0.45b0 opentelemetry-proto==1.24.0 opentelemetry-sdk==1.24.0 opentelemetry-semantic-conventions==0.45b0 opentelemetry-util-http==0.45b0 packaging==24.0 parso==0.8.4 pexpect==4.9.0 pillow==10.3.0 pipenv==2023.12.1 platformdirs==4.2.0 pluggy==1.5.0 prompt-toolkit==3.0.43 protobuf==4.25.3 psutil==5.9.8 psycopg2==2.9.9 psycopg2-binary==2.9.9 ptyprocess==0.7.0 pure-eval==0.2.2 py==1.11.0 pycodestyle==2.11.1 pycparser==2.22 pycurl==7.45.3 pydantic==2.7.0 pydantic-settings==2.2.1 pydantic_core==2.18.1 pydyf==0.9.0 pyflakes==3.2.0 Pygments==2.17.2 PyJWT==2.8.0 pylint==3.1.0 PyNaCl==1.5.0 pynliner==0.8.0 pyphen==0.14.0 pypng==0.20220715.0 pytest==7.4.4 pytest-asyncio==0.23.6 pytest-celery==1.0.0 pytest-cov==4.1.0 pytest-django==4.8.0 pytest-docker-tools==3.1.3 pytest-lazy-fixture==0.6.3 pytest-split==0.8.2 pytest-watch==4.2.0 pytest-xdist==3.3.1 python-crontab==3.0.0 python-dateutil==2.9.0.post0 python-dotenv==1.0.1 python-stdnum==1.20 pytz==2024.1 PyYAML==6.0.1 qrcode==7.4.2 redis==5.0.3 requests==2.31.0 requests-aws4auth==1.2.3 requests-mock==1.12.1 requests-oauthlib==2.0.0 requests-toolbelt==1.0.0 responses==0.25.0 retry==0.9.2 s3transfer==0.7.0 sentry-sdk==1.45.0 sepaxml==2.1.1 simple-term-menu==1.6.4 six==1.16.0 soupsieve==2.5 spider==1.0.34 sqlparse==0.5.0 stack-data==0.6.3 suds-community==1.1.2 symba==0.36 tablib==3.6.1 text-unidecode==1.3 tinycss2==1.2.1 tokipona==1.4.0 tomlkit==0.12.4 tqdm==4.66.2 traitlets==5.14.3 typeguard==2.13.3 typelog==0.3.0 typing_extensions==4.11.0 tzdata==2024.1 unittest-xml==0.2.2 unittest-xml-reporting==3.2.0 urllib3==2.0.7 uWSGI==2.0.25.1 vine==5.1.0 virtualenv==20.26.2 watchdog==4.0.0 wcwidth==0.2.13 weasyprint==61.2 webencodings==0.5.1 Werkzeug==3.0.2 wrapt==1.16.0 xlrd==2.0.1 XlsxWriter==3.2.0 xlwt==1.3.0 xmlschema==3.3.0 xmltodict==0.13.0 yarl==1.9.4 zeep==3.4.0 zipp==3.18.1 zopfli==0.2.3

pip version 24.0

Confirming that solution DD_TRACE_ASYNCIO_ENABLED=false python3 manage.py shell helped me too.

lvieirajr commented 1 month ago

I have the same issues too.