antropic subsequent requests with attached files caching

ArtemMartus commented 1 week ago

Aider version: 0.58.1 Python version: 3.12.6 Platform: Linux-6.8.0-45-generic-x86_64-with-glibc2.39 Python implementation: CPython Virtual environment: No OS: Linux 6.8.0-45-generic (64bit) Git version: git version 2.43.0

Aider v0.58.1 Main model: claude-3-5-sonnet-20240620 with diff edit format, prompt cache, infinite output Weak model: claude-3-haiku-20240307 Git repo: .git with 11,323 files Warning: For large repos, consider using --subtree-only and .aiderignore See: https://aider.chat/docs/faq.html#can-i-use-aider-in-a-large-mono-repo Repo-map: using 2048 tokens, files refresh Added subprojects/gstreamer/gst/gstbin.c to the chat. Added subprojects/gstreamer/gst/gstbin.h to the chat.

subsequent requests with attached files ignore antropic sonnet3.5 caching rules. Why does it hit only 1k tokens out of 57k if the files and system prompt are the same? It has to work correctly, fix it

ArtemMartus commented 1 week ago

Also the subtree-only option doesnt work for repo-map at all. It continues caching the whole sources folder and /map command outputs almost garbage tags relative to what I told it to map via the --subtree-only flag

fry69 commented 1 week ago

Thank you for filing this issue.

The way Anthropic's prompt caching works allows only 4 cache markers for 4 cache blocks. If anything inside these blocks change (e.g. files get edited, added or dropped with /add, /read or /drop), the block gets invalidated. Also this is prefix based, so if a previous block changes, the following blocks also get invalidated.

For more details see here -> https://aider.chat/docs/usage/caching.html And here -> https://docs.anthropic.com/en/docs/build-with-claude/prompt-caching

Also please only file one problem per issue. If you have additional problems, please file another issue.

Aider-AI / aider

antropic subsequent requests with attached files caching #1921