Runtime Exception on OpenAI vision model

[jsullivan-zonos]

Summary of problem

An exception is being raised by ddtrace when using OpenAI's vision models. This vision models accept a list of dictionaries in the content field, instead of just a string.

Minimal test case

import openai

client = openai.OpenAI()

response = client.chat.completions.create(
    model="gpt-4-vision-preview",
    messages=[
        {
            "role": "user",
            "content": [
                {"type": "text", "text": "What’s in this image?"},
                {
                    "type": "image_url",
                    "image_url": {
                        "url": "https://upload.wikimedia.org/wikipedia/commons/thumb/d/dd/Gfp-wisconsin-madison-the-nature-boardwalk.jpg/2560px-Gfp-wisconsin-madison-the-nature-boardwalk.jpg",
                    },
                },
            ],
        }
    ],
    max_tokens=300,
)

Traceback

$ ddtrace-run python test_ddtrace_openai.py 
[{'type': 'text', 'text': 'What’s in this image?'}, {'type': 'image_url', 'image_url': {'url': 'https://upload.wikimedia.org/wikipedia/commons/thumb/d/dd/Gfp-wisconsin-madison-the-nature-boardwalk.jpg/2560px-Gfp-wisconsin-madison-the-nature-boardwalk.jpg'}}]
Traceback (most recent call last):
  File "/home/jonathan/test_ddtrace_openai.py", line 5, in <module>
    response = client.chat.completions.create(
  File "/home/jonathan/miniconda3/lib/python3.9/site-packages/openai/_utils/_utils.py", line 271, in wrapper
    @functools.wraps(func)
  File "/home/jonathan/miniconda3/lib/python3.9/site-packages/ddtrace/contrib/openai/patch.py", line 401, in patched_endpoint
    g.send(None)
  File "/home/jonathan/miniconda3/lib/python3.9/site-packages/ddtrace/contrib/openai/patch.py", line 365, in _traced_endpoint
    hook.send(None)
  File "/home/jonathan/miniconda3/lib/python3.9/site-packages/ddtrace/contrib/openai/_endpoint_hooks.py", line 69, in handle_request
    self._record_request(pin, integration, span, args, kwargs)
  File "/home/jonathan/miniconda3/lib/python3.9/site-packages/ddtrace/contrib/openai/_endpoint_hooks.py", line 254, in _record_request
    content = integration.trunc(m.get("content", ""))
  File "/home/jonathan/miniconda3/lib/python3.9/site-packages/ddtrace/contrib/_trace_utils_llm.py", line 139, in trunc
    text = text.replace("\n", "\\n").replace("\t", "\\t")
AttributeError: 'list' object has no attribute 'replace'
failed to send, dropping 1 traces to intake at http://localhost:8126/v0.4/traces after 3 retries

Which version of dd-trace-py are you using?

2.3.1

Which version of pip are you using?

23.0.1

Which libraries and their versions are you using?

openai==1.3.7

`pip freeze`

How can we reproduce your problem?

See above minimal example.

What is the result that you get?

A runtime exception when using gpt-4 vision

What is the result that you expected?

No exception being raised and proper tracing for vision models.

[emmettbutler]

:eyes: @Yun-Kim

[Yun-Kim]

Hi @jsullivan-zonos, thanks for reaching out!

We've already addressed this in #7759, but this hasn't been released yet (I imagine we'll get this released in the upcoming two weeks). Once this is released in 2.4.0 or 2.3.2, please give it a try.

[emmettbutler]

Closing because this has been fixed. Please reopen if the issue persists after a version upgrade.