[BFCL] Add Prompt Caching for Claude Models

ShishirPatil / gorilla

Gorilla: Training and Evaluating LLMs for Function Calls (Tool Calls)

https://gorilla.cs.berkeley.edu/

Apache License 2.0

11.52k stars 1.01k forks source link

[BFCL] Add Prompt Caching for Claude Models #727

Closed vsvaidya27 closed 2 weeks ago

vsvaidya27 commented 1 month ago

Added prompt caching to Claude for FC and prompting models to save costs. The attached table compares the pricing for the first 15 test cases of multi-turn_base for caching and without caching for FC models (with -old indicating without caching).

ClaudeCachingandWithoutComparison

vsvaidya27 commented 3 weeks ago

Reopened PR and edited description.

HuanzhiMao commented 2 weeks ago

Close due to #751