search

simonw / llm

Access large language models from the command-line
https://llm.datasette.io
Apache License 2.0
4.85k stars 272 forks source link

OpenAI token usage stored incorrectly #614

Closed simonw closed 2 weeks ago

simonw commented 2 weeks ago

Spotted this in llm logs -c --json:

      "usage": {
        "completion_tokens": 186,
        "prompt_tokens": 48711,
        "total_tokens": 48897,
        "completion_tokens_details": "CompletionTokensDetails(audio_tokens=0, reasoning_tokens=0, accepted_prediction_tokens=0, rejected_prediction_tokens=0)",
        "prompt_tokens_details": "PromptTokensDetails(audio_tokens=0, cached_tokens=48512)"
      }

Related:

simonw commented 2 weeks ago

Now looks like this:

      "usage": {
        "completion_tokens": 359,
        "prompt_tokens": 48910,
        "total_tokens": 49269,
        "prompt_tokens_details": {
          "cached_tokens": 48768,
          "audio_tokens": 0
        },
        "completion_tokens_details": {
          "reasoning_tokens": 0,
          "audio_tokens": 0,
          "accepted_prediction_tokens": 0,
          "rejected_prediction_tokens": 0
        }
      },