Prompt caching with claude from Anthropic

lofcz / LlmTornado

One .NET library to consume OpenAI, Anthropic, Cohere, Google, Azure, Groq, and self-hosed APIs.

https://github.com/lofcz/LlmTornado

MIT License

37 stars 12 forks source link

Prompt caching with claude from Anthropic #25

Open MBrekhof opened 1 month ago

MBrekhof commented 1 month ago

Will you be adding prompt caching for anthropic (https://www.anthropic.com/news/prompt-caching)?

lofcz commented 1 month ago

Prompt caching will be added soon.

lofcz commented 1 month ago

Upon further investigation, I might implement this only after the function moves to GA. The current API for caching is prone to breaking and I'd like to avoid any unnecessary breaking changes on my side.

schaveyt commented 1 week ago

In case a you need a reference for implementation. This is the source code to the amazing AI-Assitant VSCode Extension written in Typescript. He's been using the prompt caching for a while now https://github.dev/schaveyt/cline/blob/4658e5ceadffc1361bb9604e656b2880f083cb3b/src/api/providers/anthropic.ts#L34-L35