dotnet / extensions

This repository contains a suite of libraries that provide facilities commonly needed when creating production-ready applications.
MIT License
2.63k stars 751 forks source link

[Microsoft.Extensions.AI] Add EndUserId as tag to span/traces/metrics #5583

Open kzu opened 1 week ago

kzu commented 1 week ago

It's typically quite useful to be able to associate genai telemetry with a particular user, for multiple reasons (i.e. track token consumption per user, rate limit per user, etc.). OpenAI and Claude support such an identifier specifically, but I couldn't find a similar mechanism for Gemini or Cohere.

While this is easy to do add this tag for the activity/span itself (via a delegating chat client that adds that to the Activity.Current), it's impossible to do the same for the token consumption and operation duration metrics.

It would be ideal if arbitrary (serializable/primitive?) properties in the ChatOptions.AdditionalProperties were automatically added to both traces and metrics automatically by the OpenTelemetryChatClient, or alternatively, at least just the EndUserId. It could also be an explicit setting for the client, to be configured just like EnableSensitiveData, such as EnableAdditionalProperties or EnableAdditionalTags).


Workaround: how to add EndUserId to traces (not metrics though)

    class UserIdChatClient(IChatClient client) : DelegatingChatClient(client)
    {
        public override Task<Microsoft.Extensions.AI.ChatCompletion> CompleteAsync(IList<Microsoft.Extensions.AI.ChatMessage> chatMessages, ChatOptions? options = null, CancellationToken cancellationToken = default)
        {
            if (Activity.Current is { } activity && options?.AdditionalProperties?.TryGetValue("EndUserId", out var endUserId) == true)
                activity.SetTag("user.id", endUserId);

            return base.CompleteAsync(chatMessages, options, cancellationToken);
        }

        public override IAsyncEnumerable<Microsoft.Extensions.AI.StreamingChatCompletionUpdate> CompleteStreamingAsync(IList<Microsoft.Extensions.AI.ChatMessage> chatMessages, ChatOptions? options = null, CancellationToken cancellationToken = default)
        {
            if (Activity.Current is { } activity && options?.AdditionalProperties?.TryGetValue("EndUserId", out var endUserId) == true)
                activity.SetTag("user.id", endUserId);

            return base.CompleteStreamingAsync(chatMessages, options, cancellationToken);
        }
    }

And ensure it's added right after the open telemetry client:

services.AddChatClient(builder => builder
        .UseOpenTelemetry()
        .Use(client => new UserIdChatClient(client))
        .Use(new OpenAIClient(...));
stephentoub commented 1 week ago

@lmolkova / @samsp-msft, what would you recommend here? From the perspective of adhering to the genai semantic conventions, is it ok / recommended to add additional tags?