issues
search
LangStream
/
langstream
LangStream. Event-Driven Developer Platform for Building and Running LLM AI Apps. Powered by Kubernetes and Kafka.
https://langstream.ai
Apache License 2.0
379
stars
28
forks
source link
[metrics] Add OpenAI metrics about chat completions and text completions
#684
Closed
eolivelli
closed
8 months ago
eolivelli
commented
8 months ago
Summary:
add metrics about OpenAI chat and text completions
add a script to start Prometheus and Grafana for docker run users
please note that when you are using the Streaming mode OpenAI doesn't respond yet with the usages (see this ticket
https://community.openai.com/t/openai-api-get-usage-tokens-in-response-when-set-stream-true/141866/19
)
in the "Flare" example I am disabling streaming output, this way you can see some values on Grafana (also for Flare it is useless to use Streaming output)
Summary: