search

quarkiverse / quarkus-langchain4j

Quarkus Langchain4j extension
https://docs.quarkiverse.io/quarkus-langchain4j/dev/index.html
Apache License 2.0
147 stars 88 forks source link

Observability for streaming models #1081

Open geoand opened 1 day ago

geoand commented 1 day ago

Discussed in https://github.com/quarkiverse/quarkus-langchain4j/discussions/1080

Originally posted by **cmasantos** November 14, 2024 Hi, I've notice that "normal" chat model produce a set of `gen_ai*` metrics. But I've been unable to use them with streaming models. Also when we use a combination of Mutiny StreamingResponseHandler I'm almost sure that it can't be done. My question is, how can I add metrics to a streaming model even if need to instantiate them? I've found the `MetricsChatModelListener` class but I can't find a way to put it together with the streaming model interface.

We should add observability for StreamingChatLanguageModel as well

geoand commented 1 day ago

@cmasantos would you be willing to contribute this feature?

cmasantos commented 1 day ago

@cmasantos would you be willing to contribute this feature?

Yes, I'd like to contribute. I'll give it a try, but it will take me a while because I have no idea how extensions development work. I've got to learn that first.

geoand commented 18 hours ago

Great!

In this case you probably don't need to learn too many details as the changes almost exactly match those that were made for ChatLanguageModel - I can certainly point you in the right direction.