Non-cached example appears to be cached

anthropics / anthropic-cookbook

A collection of notebooks/recipes showcasing some fun and effective ways of using Claude.

MIT License

6.74k stars 854 forks source link

Non-cached example appears to be cached #86

Closed stevekrouse closed 2 months ago

stevekrouse commented 2 months ago

Maybe I am misunderstanding how to cache things, but this "non-cached" example seems cached:

https://github.com/anthropics/anthropic-cookbook/blob/7786b9f39db8ba65202792f564c59697a5222531/misc/prompt_caching.ipynb#L160

I wasn't able to find a key difference between it and the cached version.

For context, I'm trying to convert this notebook to javascript: https://www.val.town/v/stevekrouse/anthropicCaching

tonkex commented 2 months ago

Agreed. ""cache_control": {"type": "ephemeral"}" code line in the 'make_non_cached_api_calls() function is doubtful.

alexalbertt commented 2 months ago

Ah yes it includes the cache line but on the first hit this is only a cache write, not a cache read, and thus functions like a normal API call in terms of latency.