search

BerriAI / litellm

Python SDK, Proxy Server (LLM Gateway) to call 100+ LLM APIs in OpenAI format - [Bedrock, Azure, OpenAI, VertexAI, Cohere, Anthropic, Sagemaker, HuggingFace, Replicate, Groq]
https://docs.litellm.ai/docs/
Other
13.9k stars 1.64k forks source link

[Feature]: Support cloud zero cost tracking #5773

Open krrishdholakia opened 1 month ago

krrishdholakia commented 1 month ago

The Feature

they have a feature called unit cost analytics that we were looking at using to add the info to our existing spend tracking

Motivation, pitch

our SRE team manages our other cost tracking tools (CloudZero)

Twitter / LinkedIn details

No response

krrishdholakia commented 1 month ago

Keep getting 

Failed to post telemetry data.

When running queries on their swagger 

curl --request POST \
     --url https://api.cloudzero.com/unit-cost/v1/telemetry/metric/llm-cost \
     --header 'Authorization: <my-key>' \
     --header 'content-type: application/json' \
     --data '
{
  "records": [
    {
      "value": 10,
      "timestamp": "2020-03-03T15:32:44+00:00"
    }
  ]
}
'
curl --request POST \
     --url https://api.cloudzero.com/unit-cost/v1/telemetry/metric/llm-val/sum \
     --header 'Authorization: <my-key>' \
     --header 'content-type: application/json' \
     --data '
{
  "records": [
    {
      "value": 10,
      "timestamp": "2020-03-03T15:32:44+00:00"
    }
  ]
}
'