IMplement Google Gemini Caching

AJaySi / AI-Writer

ALwrity - All in One AI Content Platform. Contextual AI Content generation for Website, Social media & Ad copywriting. Prevents AI hallucinations, Web Researched factual, SEO optimized content. Get blog Images. Create your AI Agents Content team. Generate content with RAG, Local documents, web urls, PDFs. Open Source AI writer. (WIP)

https://alwrity.com

277 stars 91 forks source link

IMplement Google Gemini Caching #91

Closed AJaySi closed 4 months ago

AJaySi commented 5 months ago

https://ai.google.dev/gemini-api/docs/caching?lang=python

AJaySi commented 4 months ago

When to use context caching Context caching is particularly well suited to scenarios where a substantial initial context is referenced repeatedly by shorter requests. Consider using context caching for use cases such as:

Chatbots with extensive system instructions Repetitive analysis of lengthy video files Recurring queries against large document sets Frequent code repository analysis or bug fixing

AJaySi commented 4 months ago

Implemented with google gemini. TBD for Openai and anthropic.

Helps save in billing, implemented for long form content. TBD for large audio & video files.