[Feature]: Support Anthropic API caching in Rivet

Ironclad / rivet

The open-source visual AI programming environment and TypeScript library

https://rivet.ironcladapp.com

MIT License

2.79k stars 244 forks source link

Open justinwetch opened 3 months ago

justinwetch commented 3 months ago

Very recently, Anthropic has added support for API caching which can save on input token costs by up to 90%. Would be a huge boon for using Claude models in Rivet. docs: https://docs.anthropic.com/en/docs/build-with-claude/prompt-caching

manojks1999 commented 3 months ago

Hi @justinwetch , can I work on this?

veloisgood commented 2 months ago

this would be extremely useful. I'm extremely interested.

abrenneke commented 2 months ago

Try running from source, I've implemented prompt caching into the latest commit