Add exponential retry logic for gemini models

ShishirPatil / gorilla

Gorilla: Training and Evaluating LLMs for Function Calls (Tool Calls)

https://gorilla.cs.berkeley.edu/

Apache License 2.0

11.52k stars 1.01k forks source link

Add exponential retry logic for gemini models #764

Closed gabrielibagon closed 1 week ago

gabrielibagon commented 1 week ago

This is to avoid the following error with long context on Gemini models due to insufficient quota:

Error: 429 Resource exhausted. Please try again later. Please refer to https://cloud.google.com/vertex-ai/generative-ai/docs/quotas#error-code-429 for more details.

This approach uses exponential backoff retries when encountering a ResourceExhausted error.