search

getsentry / sentry

Developer-first error tracking and performance monitoring
https://sentry.io
Other
39.25k stars 4.21k forks source link

AI tracking does not work properly in Python's asynchronous generator scenarios. #81227

Open uraurora opened 6 hours ago

uraurora commented 6 hours ago

Environment

SaaS (https://sentry.io/)

Steps to Reproduce

  1. First, I use an HTTP service to obtain a streaming response (SSE) from an interface, and my local interface is mainly used to relay data and report its token consumption information.
  2. Locally, I use Python FastAPI and employ a Python asynchronous generator to yield each event.
  3. I created a span within the asynchronous generator and used the decorator ai_track on the function. I used with sentry_sdk.start_span(op="ai.chat_completions.create.xxx", name="xxx") as span, and I'm not sure if the op value is set correctly.

Expected Result

I hope the LLM Monitoring works well, but seems only no-stream api does

Actual Result

The stream api does not show anything. I'm not sure whether there's an issue with my configuration or if this method of invocation is not currently supported.

Product Area

Insights

Link

https://moflow.sentry.io/insights/ai/llm-monitoring/?project=4508239351447552&statsPeriod=24h

DSN

No response

Version

2.19.0

getsantry[bot] commented 6 hours ago

Assigning to @getsentry/support for routing ⏲️