Add prompt caching when using Anthropic models

[nichochar]

Add Prompt Caching for Faster Claude Responses

Speed up Claude's response time by implementing prompt caching. See https://www.anthropic.com/news/prompt-caching. I suspect this will work really well with our Srcbook generation and edit, which have non trivial system prompts in them. This should lead to faster responses, and potential cost savings too.

Considerations

• I don't think we care about cache invalidation, but worth checking
• Measure actual performance improvements, so we can publish our findings publicly

[nichochar]

After reading further about prompt caching, I am no longer certain that this is an obvious win in all cases for Srcbook. The caching time is capped at 5mins currently (see docs), and this caching happens on a per-account basis, so a user would have to use the same prompt multiple times within a 5min span to justify paying extra on the first write.

I imagine this would be useful in 2 cases:

• generating multiple srcbooks in a row from scratch (seems unlikely)
• using AI to edit a Srcbook. This one seems more likely, but I'd like to confirm with analytics that it's justified.