[Maybe Bug] `StructuredChatAgent` raises `ValueError` with a `StructuredTool` which has an argument which defaults to a `dict` default value

[hmasdev]

System Info

OS == Windows 11 Python == 3.10.11 Langchain == 0.0.348

Who can help?

No response

Information

• [ ] The official example notebooks/scripts
• [ ] My own modified scripts

Related Components

• [ ] LLMs/Chat Models
• [ ] Embedding Models
• [ ] Prompts / Prompt Templates / Prompt Selectors
• [ ] Output Parsers
• [ ] Document Loaders
• [ ] Vector Stores / Retrievers
• [ ] Memory
• [X] Agents / Agent Executors
• [X] Tools / Toolkits
• [ ] Chains
• [ ] Callbacks/Tracing
• [ ] Async

Reproduction

Python 3.10.11 (tags/v3.10.11:7d4cc5a, Apr  5 2023, 00:38:17) [MSC v.1929 64 bit (AMD64)] on win32
Type "help", "copyright", "credits" or "license" for more information.
>>> from dotenv import load_dotenv
>>> load_dotenv()
True
>>> from langchain.agents import AgentType, initialize_agent
>>> from langchain.tools import StructuredTool
>>> from langchain.chat_models import ChatOpenAI
>>> llm = ChatOpenAI(model='gpt-3.5-turbo')
>>> def func0(a: int, b: int) -> int:
...         return a+b
...
>>> def func1(a: int, b: int, d: dict[str, int]={}) -> int:
...         print(d)
...         return a+b
...
>>> def func2(a: int, b: int, d: dict[str, int]={'c': 0}) -> int:
...         print(d)
...         return a+b
...
>>> tool0 = StructuredTool.from_function(func0, name=func0.__name__, description='a+b')
>>> tool1 = StructuredTool.from_function(func1, name=func1.__name__, description='a+b')
>>> tool2 = StructuredTool.from_function(func2, name=func2.__name__, description='a+b')
>>> agent0 = initialize_agent(agent=AgentType.STRUCTURED_CHAT_ZERO_SHOT_REACT_DESCRIPTION, tools=[tool0], llm=llm)
>>> agent1 = initialize_agent(agent=AgentType.STRUCTURED_CHAT_ZERO_SHOT_REACT_DESCRIPTION, tools=[tool1], llm=llm)
>>> agent2 = initialize_agent(agent=AgentType.STRUCTURED_CHAT_ZERO_SHOT_REACT_DESCRIPTION, tools=[tool2], llm=llm)
>>> agent0.run('hello')
'Hi there! How can I assist you today?'
>>> agent0.invoke(dict(input='hello'))
{'input': 'hello', 'output': 'Hi there! How can I assist you today?'}
>>> agent1.run('hello')
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "D:\workspace\agi-baby\venv\lib\site-packages\langchain\chains\base.py", line 507, in run
    return self(args[0], callbacks=callbacks, tags=tags, metadata=metadata)[
  File "D:\workspace\agi-baby\venv\lib\site-packages\langchain\chains\base.py", line 288, in __call__
    inputs = self.prep_inputs(inputs)
  File "D:\workspace\agi-baby\venv\lib\site-packages\langchain\chains\base.py", line 435, in prep_inputs
    raise ValueError(
ValueError: A single string input was passed in, but this chain expects multiple inputs ({'', 'input'}). When a chain expects multiple inputs, please call it by passing in a dictionary, eg `chain({'foo': 1, 'bar': 2})`
>>> agent1.invoke(dict(input='hello'))
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "D:\workspace\agi-baby\venv\lib\site-packages\langchain\chains\base.py", line 89, in invoke
    return self(
  File "D:\workspace\agi-baby\venv\lib\site-packages\langchain\chains\base.py", line 288, in __call__
    inputs = self.prep_inputs(inputs)
  File "D:\workspace\agi-baby\venv\lib\site-packages\langchain\chains\base.py", line 445, in prep_inputs
    self._validate_inputs(inputs)
  File "D:\workspace\agi-baby\venv\lib\site-packages\langchain\chains\base.py", line 197, in _validate_inputs
    raise ValueError(f"Missing some input keys: {missing_keys}")
ValueError: Missing some input keys: {''}
>>> agent2.run('hello')
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "D:\workspace\agi-baby\venv\lib\site-packages\langchain\chains\base.py", line 507, in run
    return self(args[0], callbacks=callbacks, tags=tags, metadata=metadata)[
  File "D:\workspace\agi-baby\venv\lib\site-packages\langchain\chains\base.py", line 288, in __call__
    inputs = self.prep_inputs(inputs)
  File "D:\workspace\agi-baby\venv\lib\site-packages\langchain\chains\base.py", line 435, in prep_inputs
    raise ValueError(
ValueError: A single string input was passed in, but this chain expects multiple inputs ({"'c'", 'input'}). When a chain expects multiple inputs, please call it by passing in a dictionary, eg `chain({'foo': 1, 'bar': 2})`
>>> agent2.invoke(dict(input='hello'))
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "D:\workspace\agi-baby\venv\lib\site-packages\langchain\chains\base.py", line 89, in invoke
    return self(
  File "D:\workspace\agi-baby\venv\lib\site-packages\langchain\chains\base.py", line 288, in __call__
    inputs = self.prep_inputs(inputs)
  File "D:\workspace\agi-baby\venv\lib\site-packages\langchain\chains\base.py", line 445, in prep_inputs
    self._validate_inputs(inputs)
  File "D:\workspace\agi-baby\venv\lib\site-packages\langchain\chains\base.py", line 197, in _validate_inputs
    raise ValueError(f"Missing some input keys: {missing_keys}")
ValueError: Missing some input keys: {"'c'"}

Expected behavior

Given a StructuredTool which has an argument which defaults to a dict value, StructuredChatAgent with the tool should work.

In the above reproduction codes, agent1 and agent2 should works as agent0 works.

[dosubot[bot]]

🤖

Hey @hmasdev! Good to see you again. Thanks for diving deep into the LangChain and bringing this potential bug to light. Let's get to the bottom of it!

Thank you for providing a detailed description of the issue you're encountering. It seems like the StructuredChatAgent is having trouble handling StructuredTool objects that have arguments with default dict values.

The StructuredChatAgent creates a prompt for each tool during initialization, which includes the tool's name, description, and arguments schema. The arguments schema is formatted to escape any curly braces, which might be causing the issue you're experiencing. When a dict is used as a default value, the curly braces in the dict might be causing a formatting error.

Here is the relevant code from the create_prompt method in the StructuredChatAgent class:

for tool in tools:
    args_schema = re.sub("}", "}}", re.sub("{", "{{", str(tool.args)))
    tool_strings.append(f"{tool.name}: {tool.description}, args: {args_schema}")

As you can see, the args_schema is created by converting the tool.args to a string and then escaping any curly braces. This might be causing the ValueError you're seeing when you try to run an agent with a tool that has a dict as a default argument value.

This seems like a potential bug in the LangChain framework. I recommend creating an issue in the LangChain repository detailing the problem and your findings. The maintainers of the repository should be able to provide further assistance.

In the meantime, as a workaround, you could try avoiding using dict as default argument values in your StructuredTool objects. Instead, you could use None as the default value and then check for None in your function and assign a new dict if necessary. Here's an example:

def func1(a: int, b: int, d: Optional[dict[str, int]] = None) -> int:
    if d is None:
        d = {}
    print(d)
    return a+b

I hope this helps! If you have any further questions or need more clarification, please don't hesitate to ask.

Sources

• libs/langchain/langchain/agents/structured_chat/base.py
• libs/core/tests/unit_tests/test_tools.py

---

This response is meant to be useful and save you time. It is not meant to be a precise solution, but rather a starting point for your own research.

Help me be more useful! Please leave a 👍 if this is helpful and 👎 if it is irrelevant.

If you want to continue the conversation, start your reply with @dosu-bot.

[hmasdev]

@dosu-bot

Thanks for your response!I've come to similar conclusions. But how about modifying StructuredChatOutputParser.create_prompt as follows:

AsIs

for tool in tools:
    args_schema = re.sub("}", "}}", re.sub("{", "{{", str(tool.args)))
    tool_strings.append(f"{tool.name}: {tool.description}, args: {args_schema}")

ToBe

for tool in tools:
    tool_name = re.sub("}", "}}", re.sub("{", "{{", str(tool.name)))
    tool_description  = re.sub("}", "}}", re.sub("{", "{{", str(tool.description)))
    args_schema = re.sub("}", "}}", re.sub("{", "{{", str(tool.args)))
    tool_strings.append(f"{tool_name}: {tool_description}, args: {args_schema}")

[dosubot[bot]]

🤖

Thank you for providing a detailed description of the issue you're encountering with the StructuredChatAgent class in the LangChain framework. Your proposed solution seems to be addressing the issue of string formatting in the create_prompt method. However, the error messages you're receiving suggest that the problem might be related to how the StructuredChatAgent is handling inputs, particularly when a StructuredTool has an argument that defaults to a dict value.

The error messages indicate that the StructuredChatAgent is expecting multiple inputs, but a single string input was passed in. This suggests that the StructuredChatAgent is not correctly handling the case where a StructuredTool has an argument that defaults to a dict value.

Before we proceed with your proposed solution, I would recommend checking how you're calling the StructuredChatAgent. Make sure you're passing in a dictionary of inputs if the StructuredChatAgent expects multiple inputs. For example, if your StructuredTool has an argument that defaults to a dict value, you should call the StructuredChatAgent like this:

agent = StructuredChatAgent.from_llm_and_tools(llm, tools)
result = agent.run({'foo': 1, 'bar': 2})

If you're already doing this and still encountering the issue, it would be helpful if you could provide a minimal, reproducible example of the code that's causing the error. This will help us better understand the issue and propose a more accurate solution.

Regarding your proposed solution, it seems to be addressing the string formatting in the create_prompt method. While this might help with the readability of the prompts, it's not clear how it would resolve the ValueError you're encountering. The create_prompt method is used to create a prompt for the chat, and it doesn't directly handle the inputs to the StructuredChatAgent.

If the issue persists, it might be necessary to modify the StructuredChatAgent or StructuredTool classes to better handle arguments that default to a dict value. However, without a minimal, reproducible example, it's difficult to provide a specific solution.

Please provide a minimal, reproducible example if possible, and I'll be glad to assist you further. You can find more information on how to create a minimal, reproducible example in the Python documentation.

For more information on the StructuredChatAgent and StructuredTool classes, you can refer to the LangChain documentation or the source code:

• StructuredChatAgent: libs/langchain/langchain/agents/structured_chat/base.py
• StructuredTool: libs/core/langchain_core/tools.py

I hope this helps! If you have any further questions, feel free to ask.

Dosu

Sources

• libs/langchain/langchain/agents/structured_chat/base.py
• libs/core/langchain_core/tools.py

---

This response is meant to be useful and save you time. It is not meant to be a precise solution, but rather a starting point for your own research.

Help me be more useful! Please leave a 👍 if this is helpful and 👎 if it is irrelevant.

If you want to continue the conversation, start your reply with @dosu-bot.