Open julian-risch opened 3 weeks ago
Is your feature request related to a problem? Please describe. Anthropic, in particular the models Claude 3.5 Sonnet and Claude 3 Haiku support prompt caching or context caching. Support for Claude 3 Opus is coming soon. We should enable users to use that feature through Haystack to reduce costs and latency. https://docs.anthropic.com/en/docs/build-with-claude/prompt-caching https://www.anthropic.com/news/prompt-caching
Describe the solution you'd like We need to implement support for Anthropic's cache_control parameter in messages.
cache_control
related issue https://github.com/deepset-ai/haystack-core-integrations/issues/1003
To have a full benefit of prompt caching in Haystack pipelines we need to resolve https://github.com/deepset-ai/haystack-core-integrations/issues/1011
Is your feature request related to a problem? Please describe. Anthropic, in particular the models Claude 3.5 Sonnet and Claude 3 Haiku support prompt caching or context caching. Support for Claude 3 Opus is coming soon. We should enable users to use that feature through Haystack to reduce costs and latency. https://docs.anthropic.com/en/docs/build-with-claude/prompt-caching https://www.anthropic.com/news/prompt-caching
Describe the solution you'd like We need to implement support for Anthropic's
cache_control
parameter in messages.