Lets get a sample of standard retry logic with exponential backoff, etc.

[SeaDude]

Description of the feature request:

There are many recipes for all sorts of functionality, but none (that I can find) that show retry logic for return codes 429, 503 and 500. I'm seeing these return codes A LOT.

What problem are you trying to solve with this feature?

More robust API calls.

Any other information you'd like to share?

This snippet to successfully retry when return code is 429 Resource Exhausted but times-out if return code is 503 Model is Overloaded or if 500 An internal error has occurred.

from google.generativeai.types import RequestOptions
from google.api_core import retry

def submit_gemini_query(api_key, system_message, user_message, response_class):

    genai.configure(api_key=api_key)

    generation_config = {
        "temperature": 0,
        "max_output_tokens": 8192
    }

    model = genai.GenerativeModel(
        model_name="gemini-1.5-pro-latest",
        generation_config=generation_config,
        system_instruction=system_message
    )

    response = model.generate_content(user_message,
                                      request_options=RequestOptions(
                                        retry=retry.Retry(
                                            initial=10, 
                                            multiplier=2, 
                                            maximum=60, 
                                            timeout=300
                                        )
                                       )
                                    )

    return response.text

• 
• 
•