spring-projects / spring-ai

An Application Framework for AI Engineering
https://docs.spring.io/spring-ai/reference/index.html
Apache License 2.0
3.32k stars 850 forks source link

Streaming metrics not being captured #1707

Closed habuma closed 1 week ago

habuma commented 1 week ago

Tested this with the latest snapshot.

I noticed that most of the low-cardinality tags for the "spring.ai.chat.client" metric are the same regardless of the kind of request. But the one that should vary a little is "spring.ai.chat.client.stream", which I'd assume would be true if making a streaming request and false if making a normal chat request.

But, I created a Spring AI app with a mix of streaming and non-streaming requests and I only ever see "false" for "spring.ai.chat.client.stream". It's never true. What's more, I set it up to always send streaming requests and the metric is never populated at all.

It seems that this is a bug.

tzolov commented 1 week ago

@habuma, I'm unable to reproduce it the issue. We have a various tests that ensure that the propagation low-cardinality key is propagated as expected. Also i've run a manual test where i'm combining stream and non-stream chat-client invocations and can observer metrics for both:

Screenshot 2024-11-10 at 14 38 48 Screenshot 2024-11-10 at 14 39 00

Am I missing something?

habuma commented 1 week ago

I withdraw this issue. I have tried it a few times since and it is working fine. I'm unsure what I was doing before where it wasn't working.