Open Huyueeer opened 1 month ago
@Huyueeer how do i repro this?
@Huyueeer how do i repro this?
@krrishdholakia Use Azure to build an openai model. The deployment name and model name are inconsistent. Use litellm-litellm:v1.41.11.dev1 docker deployment, enable prometheus monitoring, copy https://github.com/BerriAI/litellm/tree/main/cookbook/litellm_proxy_server/grafana_dashboard, enable monitoring in grafana, and concurrently request the Azure openai model just deployed, and you can see it.
I wonder if it has something to do with the sum by (model) (increase(litellm_requests_metric_total[5m]))
statement in grafana. I am not very familiar with this statement.
What happened?
My configuration is this:
This configuration is actually one model, not two, and the number of requests does not match.
In fact, I only made 105 requests, but in the monitoring, I saw that these two models (actually one model) made many requests respectively, but there was no connection between them.
Relevant log output
No response
Twitter / LinkedIn details
No response