SQL Agent with "llm.with_retry()": Can't instantiate abstract class BaseLanguageModel with abstract methods agenerate_prompt, apredict, apredict_messages, generate_prompt, invoke, predict, predict_messages (type=type_error)

Checked other resources

[X] I added a very descriptive title to this issue.
[X] I searched the LangChain documentation with the integrated search.
[X] I used the GitHub search to find a similar question and didn't find it.
[X] I am sure that this is a bug in LangChain rather than my code.
[X] The bug is not resolved by updating to the latest stable version of LangChain (or the specific integration package).

Example Code

import openai
from langchain_community.utilities import SQLDatabase
from langchain_community.agent_toolkits import create_sql_agent
from langchain_openai import AzureChatOpenAI
from langchain_core.prompts import (
    ChatPromptTemplate,
    MessagesPlaceholder,
)
from sqlalchemy import create_engine
import os

os.environ["AZURE_OPENAI_ENDPOINT"] = "..."
os.environ["AZURE_OPENAI_API_KEY"] = "..."
os.environ["OPENAI_API_VERSION"] = "..."

engine = create_engine("sqlite:///:memory:")
db = SQLDatabase(engine)

prompt = ChatPromptTemplate.from_messages(
        [("system", "You are a helpful agent"), ("human", "{input}"), MessagesPlaceholder("agent_scratchpad")]
    )

llm = AzureChatOpenAI(model="gpt-4", temperature=0)

llm = llm.with_retry(
    retry_if_exception_type=(openai.RateLimitError, openai.BadRequestError),
    wait_exponential_jitter=True,
    stop_after_attempt=3
)

agent_executor = create_sql_agent(llm, db=db, prompt=prompt, agent_type="openai-tools")

Error Message and Stack Trace (if applicable)

Can't instantiate abstract class BaseLanguageModel with abstract methods agenerate_prompt, apredict, apredict_messages, generate_prompt, invoke, predict, predict_messages (type=type_error)

Description

I cannot use llm.with_retry() inside an sql agent. It works fine if I don't use .with_retry()

System Info

langchain==0.2.0 langchain-community==0.2.0 langchain-core==0.2.0 langchain-openai==0.1.7 langchain-text-splitters==0.2.0

MacOS

Python Version: 3.9.18

Hi, I am also experiencing the same issue. I would like to understand the standard behavior for setting retries. In the legacy ChatOpenAI class (langchain_community/chat_models/openai.py) I see we have a retry decorator to automatically retry for the following errors: Timeout, APIError, APIConnectionError, RateLimitError, ServiceUnavailableError

I recently upgraded to use ChatOpenAI from langchain_openai. I tried using with_retry as follows:

llm = ChatOpenAI().with_retry(
    retry_if_exception_type=(openai.APIError, TypeError),
    wait_exponential_jitter=True,
    stop_after_attempt=3,
)

However, I encounter issues when providing this LLM to create_tool_calling_agent or ConversationSummaryBufferMemory. I keep getting the same error:

Can't instantiate abstract class BaseLanguageModel with abstract methods agenerate_prompt, apredict, apredict_messages, generate_prompt, invoke, predict, predict_messages (type=type_error)

@baskaryan @eyurtsev Could you please take a look at this issue? Your help would be much appreciated!

langchain-ai / langchain