Tooling to build LLM applications: prompt templating and composition, agents, LLM memory, and other instruments for builders of AI applications.
Eclipse Public License 1.0
280
stars
19
forks
source link
Handle LLM call failures #51
Open
zmedelis opened 3 months ago
Add resilient calls to LLM: rate limits, exponential backoff, etc. This needs to work across all clients: Mistral, Cohere, Claude, OAI.
Most likely to be implemented with https://github.com/KingMob/TrueGrit