feat: add LLM caching - Githubissues

ajndkr / lanarky

The web framework for building LLM microservices

https://lanarky.ajndkr.com/

MIT License

975 stars 74 forks source link

feat: add LLM caching #72

Closed ajndkr closed 1 year ago

ajndkr commented 1 year ago

Description

This PR adds the LLM caching feature for Lanarky users!

Closes #67.

The feature has been added to LangchainRouter and currently supports the following:

In-memory
Redis
GPTCache

Changelog:

✨ added LLM caching to LangchainRouter
♻️ minor refactoring to register module (towards #71 )
✅ updated unit tests
➕ added new extras
📝 updated README and docs