search

jlewi / foyle

Foyle is a copilot to help developers deploy and operate their applications.
https://foyle.io
Apache License 2.0
111 stars 10 forks source link

Monitor Prompt Caching For GPT4-o mini #315

Open jlewi opened 1 month ago

jlewi commented 1 month ago

We should monitor prompt caching in gpt4o https://platform.openai.com/docs/guides/prompt-caching

This will be useful for potentially tuning things to make better use of prompting to lower costs or improve latency.