search

Aider-AI / aider

aider is AI pair programming in your terminal
https://aider.chat/
Apache License 2.0
22.23k stars 2.07k forks source link

Incorrect tokens logging occasionally #2274

Closed dopper closed 1 week ago

dopper commented 1 week ago

Issue

When sending content using sonnet I have noticed that often but not always the number of tokens sent appears to be double. When looking at the logs in the API client it shows the correct amount. Here is a recent example:

In this case the 108k cache write matches the tokens sent, but Aider is doubling this value which produces and incorrect estimate cost. I think this just started happening after 0.62.

Version and model info

Aider v0.62.1 Main model: claude-3-5-sonnet-20241022 with architect edit format, prompt cache, infinite output Editor model: claude-3-5-sonnet-20241022 with editor-diff edit format Weak model: claude-3-5-haiku-20241022 Git repo: .git with 736 files Repo-map: using 2048 tokens, files refresh

paul-gauthier commented 1 week ago

Thanks for trying aider and filing this issue.

How do you know only 108k were sent?

dopper commented 1 week ago

Hi, I was comparing token output from Aider to logs from LLM provider to look for discrepancies.

paul-gauthier commented 1 week ago

Ok, thanks. I will look into this.

This looks like a duplicate of #2304. Please see the comments there for more information, and feel free to continue the discussion within that issue.

I'm going to close this issue for now. But please let me know if you think this is actually a distinct issue and I will reopen this issue.