Pandas agent got error:Prompt missing required variables: {'tool_names', 'tools'}

Xls1994 commented 4 months ago

Checked other resources

[X] I added a very descriptive title to this issue.
[X] I searched the LangChain documentation with the integrated search.
[X] I used the GitHub search to find a similar question and didn't find it.
[X] I am sure that this is a bug in LangChain rather than my code.

Example Code

import pandas as pd from langchain_experimental.agents.agent_toolkits import create_pandas_dataframe_agent from typing import Any, List, Mapping, Optional from langchain_core.callbacks.manager import CallbackManagerForLLMRun from langchain_core.language_models.llms import LLM class CustomLLM(LLM): n: int

@property
def _llm_type(self) -> str:
    return "custom"

def _call(
    self,
    prompt: str,
    stop: Optional[List[str]] = None,
    run_manager: Optional[CallbackManagerForLLMRun] = None,
    **kwargs: Any,
) -> str:
    if stop is not None:
        raise ValueError("stop kwargs are not permitted.")
    return prompt[: self.n]

@property
def _identifying_params(self) -> Mapping[str, Any]:
    """Get the identifying parameters."""
    return {"n": self.n}

df = pd.read_csv("./titanic.csv") llm=CustomLLM(n=10) agent = create_pandas_dataframe_agent(llm,df,verbose=True)

result = agent.run("这艘船上存活的男性有多少人")

print(result)

Error Message and Stack Trace (if applicable)

Traceback (most recent call last): File "/langchain_demos.py", line 30, in agent = create_pandas_dataframe_agent(llm,df,verbose=True,input_variables=input_vars) File "/Users/yyl/opt/anaconda3/envs/ragpy310/lib/python3.10/site-packages/langchain_experimental/agents/agent_toolkits/pandas/base.py", line 264, in create_pandas_dataframe_agent runnable=create_react_agent(llm, tools, prompt), # type: ignore File "/Users/yyl/opt/anaconda3/envs/ragpy310/lib/python3.10/site-packages/langchain/agents/react/agent.py", line 97, in create_react_agent raise ValueError(f"Prompt missing required variables: {missing_vars}") ValueError: Prompt missing required variables: {'tool_names', 'tools'}

Description

I'm trying to use the pandas agent function from Langchain and encountering this error.

System Info

langchain 0.1.5 langchain-community 0.0.17 langchain-core 0.1.18 langchain-experimental 0.0.50

macos python 3.10.13

dosubot[bot] commented 4 months ago

🤖

Hey @Xls1994, great to see you back! I hope you've been doing well.

The error you're encountering is due to the create_pandas_dataframe_agent function in LangChain expecting certain variables in the prompt that are not being provided. Specifically, it's looking for tool_names and tools variables in the prompt, which are not present in your code.

The create_pandas_dataframe_agent function is part of LangChain's experimental features and is used to create an agent that can interact with a pandas DataFrame. It uses a language model (LLM) and a DataFrame to generate responses based on the input prompt.

The function works by creating a RunnableAgent with a create_react_agent runnable. This runnable requires a prompt that includes certain variables, including tool_names and tools. These variables are used to define the tools that the agent can use to interact with the DataFrame.

In your code, you're passing a string as the prompt to the agent.run method. This string doesn't include the required tool_names and tools variables, which is why you're seeing the error.

To fix this, you need to include these variables in your prompt. Here's an example of how you can do this:

prompt = {
    "tool_names": ["pandas"],
    "tools": {
        "pandas": {
            "df": df.to_dict()
        }
    },
    "input": "这艘船上存活的男性有多少人"
}

result = agent.run(prompt)

In this example, tool_names is a list of the names of the tools that the agent can use. tools is a dictionary where the keys are the names of the tools and the values are dictionaries that define the state of the tools. For the pandas tool, the state is defined by the DataFrame, which is converted to a dictionary using the to_dict method.

Please note that this is a simplified example and you may need to adjust it based on the specifics of your use case.

I hope this helps! If you have any other questions, feel free to ask.

Sources

This response is meant to be useful and save you time. It is not meant to be a precise solution, but rather a starting point for your own research.

Help me be more useful! Please leave a 👍 if this is helpful and 👎 if it is irrelevant.

If you want to continue the conversation, start your reply with @dosu-bot.

Xls1994 commented 4 months ago

This should be caused by code not being updated or missed. between the langchain_core and langchain_experimental

alvaradojl commented 4 months ago

I have exactly the same problem when creating the agent with create_pandas_dataframe_agent and Azure Open AI

tadeodonegana commented 4 months ago

Same problem here.

Downgrading to langchain-experimental==0.0.49 solve the problem temporarily.

hulk66 commented 4 months ago

also, as a terrible ugly hack, in langchain_experimental/agents/agent_toolkits/pandas/base.py in create_pandas_dataframe_agentadding the two lines

        prompt.input_variables.append("tools")
        prompt.input_variables.append("tool_names")

before

        agent: Union[BaseSingleActionAgent, BaseMultiActionAgent] = RunnableAgent(
            runnable=create_react_agent(llm, tools, prompt),  # type: ignore
            input_keys_arg=["input"],
            return_keys_arg=["output"],
        )

overcomes the symptom but most likely not the cause.

usaa7 commented 4 months ago

Brother I am beginner and having a same issue while calling create_csv_agent(create_csv_agent(llm, user_csv, verbose=True)) Please tell me how you fixed it ?

RobertSelvam commented 4 months ago

agent_chain = create_csv_agent(

File "/home/user/.local/lib/python3.10/site-packages/langchain_experimental/agents/agent_toolkits/csv/base.py", line 66, in create_csv_agent return create_pandas_dataframe_agent(llm, df, **kwargs) File "/home/user/.local/lib/python3.10/site-packages/langchain_experimental/agents/agent_toolkits/pandas/base.py", line 264, in create_pandas_dataframe_agent runnable=create_react_agent(llm, tools, prompt), # type: ignore File "/home/user/.local/lib/python3.10/site-packages/langchain/agents/react/agent.py", line 97, in create_react_agent raise ValueError(f"Prompt missing required variables: {missing_vars}") ValueError: Prompt missing required variables: {'tool_names', 'tools'}

Maple195 commented 4 months ago

Having the same issue. Please help!!

usaa7 commented 4 months ago

agent_chain = create_csv_agent(
File "/home/user/.local/lib/python3.10/site-packages/langchain_experimental/agents/agent_toolkits/csv/base.py", line 66, in create_csv_agent return create_pandas_dataframe_agent(llm, df, **kwargs) File "/home/user/.local/lib/python3.10/site-packages/langchain_experimental/agents/agent_toolkits/pandas/base.py", line 264, in create_pandas_dataframe_agent runnable=create_react_agent(llm, tools, prompt), # type: ignore File "/home/user/.local/lib/python3.10/site-packages/langchain/agents/react/agent.py", line 97, in create_react_agent raise ValueError(f"Prompt missing required variables: {missing_vars}") ValueError: Prompt missing required variables: {'tool_names', 'tools'}

Same like my issue.

broepke commented 4 months ago

I'm seeing a similar issue but i'm using ChatPromptTemplate.from_messages which doesn't use a dictionary but rather the tuple below... Any help?

ValueError: Prompt missing required variables: {'tool_names', 'tools'}

 # Generate LLM response
    prompt = ChatPromptTemplate.from_messages(
        [
            (
                "system",
                "You are a helpful assistant .... and my other prompt stuff here..."
            ),
            MessagesPlaceholder(variable_name="history"),
            ("human", "{input}"),
        ]
    )

benstager commented 4 months ago

same issue. OP response does not help. what's going on

elkanjeremy commented 4 months ago

I think the latest version has this bug. I have solved this by downgrading langchain packages. Here are the package combination that works:

langchain==0.1.3
langchain-core==0.1.15
langchain-google-vertexai
langchain-google-genai
langchain-community
langchain-experimental==0.0.49

ibarrien commented 4 months ago

@elkanjeremy: langchain-experimental==0.0.49 seems to do the trick. Versions that helped me so far:

openai==1.12.0
langchain==0.1.3
langhchain-core==0.1.22
langchain_experimental==0.1.3

NikhilKosare commented 4 months ago

facing the same error with create_sql_agent

llm = AzureChatOpenAI() db = SQLDatabase.from_uri(connection_uri)

agent = create_sql_agent( llm=llm, db=db, prompt=full_prompt, verbose=True )

ValueError: Prompt missing required variables: {'tool_names', 'tools'}

avisionh commented 2 months ago

I think this might help: https://github.com/langchain-ai/langchain/issues/17939#issuecomment-2081660241

YinSonglin1997 commented 1 month ago

I think another solution. I write the template and use prompt = PromptTemplate.from(template=template).

langchain-ai / langchain