DataDog: instrumentation alters OpenAI Async interface behaviour

otwieracz commented 1 month ago

Summary of problem

When instrumented with DataDog's patch_all() or ddtrace-run, LangFuse wrapper code throws unexpected exception when awaited with asyncio.wait_for:

AttributeError: 'NoneType' object has no attribute '__dict__'. Did you mean: '__dir__'?

Which version of dd-trace-py are you using?

[[package]]
name = "ddtrace"
version = "2.10.3"

Which version of pip are you using?

$ pip --version
pip 24.0 from /Users/slawekgonet/Work/workspace/.venv/lib/python3.12/site-packages/pip (python 3.12)

Which libraries and their versions are you using?

 $ pip freeze | egrep '(langfuse|openai|dd|async)'
async-lru==2.0.4
asyncio==3.4.3
ddsketch==3.0.1
ddtrace==2.10.3
langfuse==2.42.1
nest-asyncio==1.6.0
openai==1.40.0

How can we reproduce your problem?

import asyncio
import logging
import os
import dotenv
import ddtrace
from langfuse.openai import AsyncOpenAI
from langfuse.decorators import observe

dotenv.load_dotenv(".env.local")

logger = logging.getLogger()

async_client = AsyncOpenAI(api_key=os.getenv("OPENAI_API_KEY"))

@observe(name="error_test")
@ddtrace.tracer.wrap("error_test")
async def test():
    try:
        await asyncio.wait_for(
            async_client.chat.completions.create(
                model="gpt-4o-mini",
                messages=[
                    {
                        "role": "user",
                        "content": "Write a Python program that writes a Python program that writes the original Python program.",
                    }
                ],
            ),
            timeout=1,
        )
    except asyncio.TimeoutError:
        print("Timeout")
    except Exception as e:
        logger.exception(e)

What is the result that you get?

When run with DataDog instrumentation

ddtrace.patch_all()
await test()

---

'NoneType' object has no attribute '__dict__'
Traceback (most recent call last):
  File "/var/folders/jg/1m2fyyh10fg686tfthq84xh80000gn/T/ipykernel_37102/735597542.py", line 21, in test
    await asyncio.wait_for(
  File "/opt/homebrew/Cellar/python@3.12/3.12.4/Frameworks/Python.framework/Versions/3.12/lib/python3.12/asyncio/tasks.py", line 520, in wait_for
    return await fut
           ^^^^^^^^^
  File "/Users/slawekgonet/Work/workspace/.venv/lib/python3.12/site-packages/langfuse/openai.py", line 558, in _wrap_async
    raise ex
  File "/Users/slawekgonet/Work/workspace/.venv/lib/python3.12/site-packages/langfuse/openai.py", line 536, in _wrap_async
    openai_response.__dict__ if _is_openai_v1() else openai_response,
    ^^^^^^^^^^^^^^^^^^^^^^^^
AttributeError: 'NoneType' object has no attribute '__dict__'. Did you mean: '__dir__'?

What is the result that you expected?

When run without instrumentations:

await test()

---

Timeout

emmettbutler commented 1 month ago

Thank you @otwieracz. We'll look into it.

cc @Yun-Kim

otwieracz commented 1 month ago

I've updated the title to reflect most recent findings. This issue is not related to LangFuse in any way, but instead DataDog instrumentation changes the behaviour of AsyncOpenAI client.