Closed Kav-K closed 8 months ago
I got the same error. not sure whether it's a new bug introduced.
I think it was this commit that caused the issue, namely the concept of message lists. When i roll back to before the PR was committed the error is gone.
@hwchase17 any insights on how to fix the bug to make it compatible with the current version?
facing this exact same issue using chat conversational react description
Have this same issue. this needs to be prioritized, conversational agents are absolutely broken for any sort of robust use case.
I have the same error in the ConversationalChatAgent with most of my inputs.
I have a pull request out for this.
The issue is that the agent doesn't always respond with proper markdown which makes the JSON invalid OR it doesn't include the json
at the beginning of the markdown.
You can hot patch this by adding this output parser and then calling this new output parser as an argument. You'll need to do something like this to include hotpatch (FYI didn't test this code below so YMMV):
prompt = CharacterChatAgent.create_prompt(
tools,
output_parser=new_output_parser,
input_variables=["human_input", "chat_history", "agent_scratchpad"],
)
llm_chain = LLMChain(llm=ChatOpenAI(
temperature=0.7), prompt=prompt)
agent = CharacterChatAgent(
llm_chain=llm_chain, tools=tools, verbose=True, output_parser=new_output_parser)
agent_chain = AgentExecutor.from_agent_and_tools(
agent=agent,
tools=tools,
memory=memory,
)
Just re-read the original issue and recalled that when fixing this, I may have done more than just update the output_parser.
Refer to my other PR #4411 for a new chat agent. It adds additional functionality of a character's voice, but I also fixed many of these issues by modifying the prompt.
I'm facing exactly the same issue. If the max_iterations is greater than 1, the agent raises ValueError
raise ValueError(
ValueError: variable agent_scratchpad should be a list of base messages, got {
"action": "Conversation Knowledgebase",
"action_input": "Can you please be more specific about what you need help with in the game?"
}
@treppers
I have a pull request out for this.
The issue is that the agent doesn't always respond with proper markdown which makes the JSON invalid OR it doesn't include the
json
at the beginning of the markdown.You can hot patch this by adding this output parser and then calling this new output parser as an argument. You'll need to do something like this to include hotpatch (FYI didn't test this code below so YMMV):
I tried your patch with the new ConvoOutputParser class; however, I'm still getting the same error:
File "/home/gene/endpoints/venv/lib/python3.10/site-packages/langchain/prompts/chat.py", line 43, in format_messages
raise ValueError(
ValueError: variable agent_scratchpad should be a list of base messages, got {
"action": "Price Lookup",
"action_input": "Marble Madness"
}
Interestingly, on the latest langchain, this error only occurs when hitting the early_stop (ie. max_iterations=1) and early_stopping_method="generate", when it hits the stop:
Thought:
I now need to return a final answer based on the previous steps:
However, using the default early_stopping_method="force" causes no error and just returns the default string informing that the agent hit the max iterations.
This issue still exists today, early stopping + generate seems to be completely broken
still exists
I can confirm the issue still exists in version 0.0.302
I think this has been fixed now.
i don't think it has been fixed. Tried out latest version 0.1.0. Still getting the same error when using early_stopping_method="generate" and max_iteration. Can anybody help with the patch ?
I am using 0.1.14 and still facing this issue.
agent_scratchpad
is being constructed as string but a list of messages is expected.
yeah this is still super disgusting and absolutely broken
On Wed, Apr 3, 2024 at 11:39 PM Abdul Fatah @.***> wrote:
I am using 0.1.14 and still facing this issue. agent_scratchpad is being constructed as string but a list of messages is expected.
— Reply to this email directly, view it on GitHub https://github.com/langchain-ai/langchain/issues/3448#issuecomment-2036114699, or unsubscribe https://github.com/notifications/unsubscribe-auth/AFBOJDJZ4VMXDU6NEJZG4KLY3TDPXAVCNFSM6AAAAAAXJZJXI2VHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDAMZWGEYTINRZHE . You are receiving this because you modified the open/close state.Message ID: @.***>
Having the same issue. @Kav-K have you closed the issue?
Also still seeing the issue with v0.1.19 when following the agents quickstart tutorial:
ValueError: variable agent_scratchpad should be a list of base messages, got
Also still seeing the issue with v0.1.19 when following the agents quickstart tutorial:
ValueError: variable agent_scratchpad should be a list of base messages, got
I initiated my agent as following which seems to have fixed it for me
from langchain.agents.format_scratchpad import format_log_to_str
agent = (
{
"input": lambda x: x["input"],
"agent_scratchpad": lambda x: format_log_to_str(x["intermediate_steps"]),
}
| prompt
| chat_model_with_stop
| output_parser
)
Also what I noticed is that the error occurs when I pull the prompt from the hub.
prompt = hub.pull("hwchase17/react-chat-json")
and not when i hand copy/paste from the site manually and do this.
prompt = PromptTemplate.from_template(definition_prompts)
prompt = prompt.partial(
tools=render_text_description(tools),
tool_names=", ".join([t.name for t in tools]),
)
If you face this error when creating your agent with create_react_agent
, note that the chain in this function formats the agent_scratchpad as a string. If you construct your prompt manually, pay attention to define this variable as a simple prompt template variable and not as a message placeholder, i.e. use something like ("human", "{agent_scratchpad")
instead of MessagesPlaceholder(variable_name="agent_scratchpad")
.
Most of the time the agent can't parse it's own tool usage.