search

simonw / llm-claude-3

LLM plugin for interacting with the Claude 3 family of models
Apache License 2.0
254 stars 23 forks source link

add prompt caching #13

Closed irthomasthomas closed 2 months ago

irthomasthomas commented 2 months ago

Added option cache_prompt and cache_system

To use 

llm -m claude-3.5-sonnet --system $(cat long-system.txt) "prompt" -o cache_system 1
"usage": {"input_tokens": 10000, "output_tokens": 500, "cache_creation_input_tokens": 10000, "cache_read_input_tokens": 0}}
llm -m claude-3.5-sonnet --system $(cat long-system.txt) "prompt" -o cache_system 1
 "usage": {"input_tokens": 10001, "output_tokens": 500, "cache_creation_input_tokens": 0, "cache_read_input_tokens":10000 }}
llm -m claude-3.5-sonnet --system $(cat long-system.txt) "$(cat long-prompt.txt)" -o cache_system 1 -o cache_prompt 1
 "usage": {"input_tokens": 15000 "output_tokens": 500, "cache_creation_input_tokens": 5000, "cache_read_input_tokens":10000 }}
llm -m claude-3.5-sonnet --system $(cat long-system.txt) "$(cat long-prompt.txt)" -o cache_system 1 -o cache_prompt 1
 "usage": {"input_tokens": 15000 "output_tokens": 500, "cache_creation_input_tokens": 0, "cache_read_input_tokens":15000 }}