Error: 'OpenAICallbackHandler' object has no attribute 'on_retry'`

ArslanKAS commented 1 year ago

System Info

LangChain: 0.0.248 Python: 3.10.10 OS version: Linux 5.10.178-162.673.amzn2.x86_64

Who can help?

@agola11

Information

[ ] The official example notebooks/scripts
[X] My own modified scripts

Related Components

[ ] LLMs/Chat Models
[ ] Embedding Models
[ ] Prompts / Prompt Templates / Prompt Selectors
[ ] Output Parsers
[ ] Document Loaders
[ ] Vector Stores / Retrievers
[ ] Memory
[ ] Agents / Agent Executors
[ ] Tools / Toolkits
[ ] Chains
[X] Callbacks/Tracing
[ ] Async

Reproduction

Code:

            try:
                with get_openai_callback() as cb:
                    llm_chain = LLMChain(llm=llm, prompt=prompt_main)
                    all_text = str(template) + str(prompt) + str(usescases) + str(transcript)
                    threshold = (llm.get_num_tokens(text=all_text) + 800)
                    dataframe_copy.loc[index, "Total Tokens"] = threshold
                    if int(threshold) <= 4000:
                        chatgpt_output = llm_chain.run({"prompt":prompt, "use_cases_dictionary":usescases, "transcript":transcript})
                        chatgpt_output = text_post_processing(chatgpt_output)
                        dataframe_copy.loc[index, "ChatGPT Output"] = chatgpt_output
                        dataframe_copy.loc[index, "Cost (USD)"] = cb.total_cost
                    else:
                        dataframe_copy.loc[index, "ChatGPT Output"] = " "
                        dataframe_copy.loc[index, "Cost (USD)"] = " "
            except Exception as e:
                dataframe_copy.loc[index, "ChatGPT Output"] = " "
                dataframe_copy.loc[index, "Cost (USD)"] = " "
                continue

Error Message: Retrying langchain.chat_models.openai.ChatOpenAI.completion_with_retry.<locals>._completion_with_retry in 4.0 seconds as it raised APIError: Bad gateway. {"error":{"code":502,"message":"Bad gateway.","param":null,"type":"cf_bad_gateway"}} 502 {'error': {'code': 502, 'message': 'Bad gateway.', 'param': None, 'type': 'cf_bad_gateway'}} {'Date': 'Mon, 31 Jul 2023 20:24:53 GMT', 'Content-Type': 'application/json', 'Content-Length': '84', 'Connection': 'keep-alive', 'X-Frame-Options': 'SAMEORIGIN', 'Referrer-Policy': 'same-origin', 'Cache-Control': 'private, max-age=0, no-store, no-cache, must-revalidate, post-check=0, pre-check=0', 'Expires': 'Thu, 01 Jan 1970 00:00:01 GMT', 'Server': 'cloudflare', 'CF-RAY': '7ef889a50eaca7f3-SYD', 'alt-svc': 'h3=":443"; ma=86400'}. Error in OpenAICallbackHandler.on_retry callback: 'OpenAICallbackHandler' object has no attribute 'on_retry' bug

Expected behavior

I went through the callback documentation and yes the "on_retry" method wasn't included over there. So I guess the team needs to modify the core code for OpenAICallbackHandler because it's calling "on_retry" for some reason.

dosubot[bot] commented 1 year ago

Answer generated by a 🤖

Answer

Thank you for your detailed report. It seems like you're encountering an error because the OpenAICallbackHandler object is trying to call an on_retry method that doesn't exist.

Based on the context provided, the OpenAICallbackHandler in the LangChain codebase is used to handle the responses from the language model (LLM). It keeps track of the number of successful requests, the total number of tokens used, the number of prompt tokens, and the number of completion tokens. It also calculates the total cost of the requests. However, it does not have an on_retry method.

The on_retry method is typically used to handle and trace retry attempts during the execution of a run. When a retry attempt is made, this method is called with the RetryCallState object, which contains information about the retry attempt, such as how long it has been idle for and the attempt number. This method is important for understanding the execution flow of a run, especially in cases where retries are necessary due to failures or exceptions.

In your case, it seems like the OpenAICallbackHandler is trying to call the on_retry method, which is not defined in its class. This could be due to a bug in the code or a misconfiguration.

As you suggested, the core code for OpenAICallbackHandler might need to be modified to include the on_retry method. However, this would require a thorough understanding of the LangChain codebase and the specific requirements of the OpenAICallbackHandler.

I would recommend reaching out to the LangChain team or the community for further assistance. They might be able to provide a more accurate solution or workaround for this issue.

Please note that this analysis is based on the context provided and the information available in the LangChain codebase. There might be other factors at play that are not covered in this response.

Sources

This response is meant to be useful, save you time, and share context. It is not meant to be a precise solution, but rather a starting point for your own research.

Help me be more useful! Please leave a 👍 if this is helpful and 👎 if it is irrelevant.

akanshajainn commented 1 year ago

Have we found any fix for this?

landonsilla commented 1 year ago

Can't we just add a "do nothing" method for on_retry just so that it's found and doesn't error? https://github.com/langchain-ai/langchain/pull/9215 Then, subclasses can implement this if they want, or they can leave this as is and it will occur silently, without error.

langchain-ai / langchain