Closed LewisVille-flow closed 1 month ago
Maybe 'retry' could not fix this bug, already ChatGoogleGenerativeAI
has the default value for the max_retries:
https://api.python.langchain.com/en/latest/chat_models/langchain_google_genai.chat_models.ChatGoogleGenerativeAI.html#langchain_google_genai.chat_models.ChatGoogleGenerativeAI.max_retries
May 677965f
commit fix this issue, assuming one retry could fix the model overload issue.
If it happens again, I would re open this issue.
Your Envirionment
Description
Found this error,
503 The model is overloaded. Please try again later.
This error rarely occurs but it immediately cause the socket disconnection, which could lose user's input and chat history.This error is mainly caused by
google.generativeai
, and this is called with chatbot for search, because of this line below, first: https://github.com/lingo-chat/prompt/blob/9774ca2fb37c267be394dcf442d4833ae9134961/Lingo_Chat/ai_server/graph/utils.py#L10and second: https://github.com/lingo-chat/prompt/blob/9774ca2fb37c267be394dcf442d4833ae9134961/Lingo_Chat/ai_server/ws/handler.py#L51-L52
The first code(gemini llm occurs 503 error response, and then consequently handler raise the error. so, the valid approach to fix this bug should be retrying the calling gemini api.
etc(optional)
See https://stackoverflow.com/questions/78154047/encountering-503-error-when-calling-gemini-api-from-google-colab