search

sashabaranov / go-openai

OpenAI ChatGPT, GPT-3, GPT-4, DALL·E, Whisper API wrapper for Go
Apache License 2.0
8.7k stars 1.32k forks source link

OpenTelemetry instrumentation for go-openai #634

Open nirga opened 6 months ago

nirga commented 6 months ago

Is your feature request related to a problem? Please describe. I'm the maintainer of OpenLLMetry where we built a set of extensions of OpenTelemetry for GenAI applications. We currently support Python and Typescript (where we can easily monkey-patch the libraries within our instrumentations) and want to extend this to Go. Because there isn't a way to easily monkey-patch code in Go, we currently force our users to do it manually and I'd love to add this natively to your SDK so it will happen automagically.

Describe the solution you'd like An ability for a user to turn on go opentelemetry instrumentation which uses our semantic convention. Then, the user can connect it to any observability tools they want - Datadog, Honeycomb, Traceloop, etc.

Additional context Willing to do the PR and the work, just wanted a buy-in from the team before starting to work.

sashabaranov commented 6 months ago

Hey @nirga, that sounds like a really nice-to-have feature! Before we go to the implementation stage, what's your vision on how end-user API could look like?

nirga commented 6 months ago

@sashabaranov that's a good question! I was thinking to have the user enable tracing by calling some function on the openai client object like - 

client := openai.NewClient("token")
tracer := traceloop.NewTracer()
client.enableTracing(tracer)

Or it can even be when creating the client

sashabaranov commented 6 months ago

@nirga sounds good! Let's just make sure that the tracer is passed as an interface so it would be possible to instrument with other libraries, too.

nirga commented 6 months ago

I'll get to work on this this week @sashabaranov Is it ok if I use our DTO for describing messages / prompts / etc.?

sashabaranov commented 6 months ago

@nirga Sorry, I'm not sure I 100% understand the question.

I think that the client<->tracer interface should not lose any information and be ergonomic at the same time.

In terms of "not losing information", — we can pass complete requests (JSON-encoded strings?) and a tracer can take all the information it needs. We can even use HTTP tracing https://go.dev/blog/http-tracing — it's possible to specify custom HTTP client/transport with this library (see proxying example in the README).

But I'm not sure whether this would be useful and ergonomic for tracers to use — would love your input here!