Closed gcsun closed 7 months ago
I got the same error while trying gpt-3.5-turbo
model with Python REPL tool. But when I changed my model the tool was used.
I am using langchain version 0.0.107
I'm having the same issue when using gpt-3.5-turbo
and langchain 0.0.118
. This is what I receive:
Observation: Use the `value_counts()` method on the "Hoe voel je je?" column to count the number of occurrences of each value. is not a valid tool, try another one.
Anyone who has an idea on how to fix this?
Same problem here. Trying to use gpt-3.5-turbo, it works with the regular model.
Langchain v 0.0.123
Code:
from langchain.agents import load_tools
from langchain.agents import initialize_agent
from langchain.chat_models import ChatOpenAI
from langchain.llms import OpenAI
llm = ChatOpenAI(temperature=0,model_name='gpt-3.5-turbo')
tools = load_tools(["python_repl"], llm=llm)
agent = initialize_agent(tools, llm, agent="zero-shot-react-description", verbose=True)
agent.run("Can i print a number in python?")
Error:
Same problem hereTraceback (most recent call last):
File "C:\Users\Name\Desktop\project\test.py", line 10, in <module>
agent.run("Can i print a number in python?")
File "C:\Users\Name\AppData\Roaming\Python\Python311\site-packages\langchain\chains\base.py", line 213, in run
return self(args[0])[self.output_keys[0]]
^^^^^^^^^^^^^
File "C:\Users\Name\AppData\Roaming\Python\Python311\site-packages\langchain\chains\base.py", line 116, in __call__
raise e
File "C:\Users\Name\AppData\Roaming\Python\Python311\site-packages\langchain\chains\base.py", line 113, in __call__
outputs = self._call(inputs)
^^^^^^^^^^^^^^^^^^
File "C:\Users\SimonPC\AppData\Roaming\Python\Python311\site-packages\langchain\agents\agent.py", line 505, in _call
next_step_output = self._take_next_step(
^^^^^^^^^^^^^^^^^^^^^
File "C:\Users\Name\AppData\Roaming\Python\Python311\site-packages\langchain\agents\agent.py", line 409, in _take_next_step
output = self.agent.plan(intermediate_steps, **inputs)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "C:\Users\Name\AppData\Roaming\Python\Python311\site-packages\langchain\agents\agent.py", line 105, in plan
action = self._get_next_action(full_inputs)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "C:\Users\Name\AppData\Roaming\Python\Python311\site-packages\langchain\agents\agent.py", line 67, in _get_next_action
parsed_output = self._extract_tool_and_input(full_output)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "C:\Users\Name\AppData\Roaming\Python\Python311\site-packages\langchain\agents\mrkl\base.py", line 138, in _extract_tool_and_input
return get_action_and_input(text)
^^^^^^^^^^^^^^^^^^^^^^^^^^
File "C:\Users\Name\AppData\Roaming\Python\Python311\site-packages\langchain\agents\mrkl\base.py", line 46, in get_action_and_input
raise ValueError(f"Could not parse LLM output: `{llm_output}`")
ValueError: Could not parse LLM output: `I need to open a Python shell to use the print() function.
Action: Open a Python shell`
I got it working by using chat-zero-shot-react-description
as agent instead of zero-shot-react-description
As far as i know, that's not documented anywhere.
from langchain.chat_models import ChatOpenAI
from langchain.agents import load_tools, initialize_agent
llm = ChatOpenAI(temperature=0)
tools = load_tools(["python_repl"])
agent= initialize_agent(tools, llm, agent="chat-zero-shot-react-description", verbose=True)
agent.run("Can i print a number in python?")
I keep getting the same issue, even when using chat-zero-shot-react-description as agent instead of zero-shot-react-description. There should be an alternative way to handle this
Mine also likes to say "Action: Use {tool_name}" instead of "Action {tool_name}", and it then gets stuck in an infinite loop trying until my context runs out
same issue with gpt-4 and gpt-3.5-turbo both, even though the tool it selected is correct, the agent says it is not a valid tool.
Just to add, the latest gpt4all model has the same problem with agent:
from langchain.llms import GPT4All
from langchain.agents import load_tools
from langchain.agents import initialize_agent
from langchain.agents import AgentType
import os
os.environ["GOOGLE_CSE_ID"] = "xxxx"
os.environ["GOOGLE_API_KEY"] = "xxxx"
llm = GPT4All(model="./model/gpt4all-lora-quantized-ggjt.bin")
toolkit = load_tools(["google-search"])
agent = initialize_agent(toolkit, llm, agent="zero-shot-react-description", verbose=True, return_intermediate_steps=True)
response = agent({"input":"what was the first album of the band that Natalie Bergman is a part of?"})
I am using gpt-3.5-turbo, work totally fine when set the agent to be like this:
agent = initialize_agent(tools,
llm,
agent="chat-zero-shot-react-description",
verbose=True)
after that the agent can choose right tool to use in the specific cases
That did it! Thanks!
I am using gpt-3.5-turbo, work totally fine when set the agent to be like this:
agent = initialize_agent(tools, llm, agent="chat-zero-shot-react-description", verbose=True)
after that the agent can choose right tool to use in the specific cases
Yeah I've did this but it still sometimes wants to use "Use
I ran into this issue. When inspecting the langchain code, in my case it was being thrown by this line: https://github.com/hwchase17/langchain/blob/bee59b4689fe23dce1450bde1a5d96b0aa52ee61/langchain/agents/agent.py#L760
After digging deeper it seemed to be thrown if the custom tool had a different class name (excluding whitespace) to what it's name
attribute is. For example, this would work:
class MyCustomTool(BaseTool):
name = "My Custom Tool"
whereas this would throw the error
class MyCustomTool(BaseTool):
name = "My Interesting new Custom Tool"
Try changing them to match and see if that helps!
I think it's a matter of the gpt model performance. When the model doens't understand and complete exactly the same as what we want, e.g. upper case vs lower case, paraphrase of the name, the procedure just doesn't move on to the next step (since now it requires the action should be exactly the same as one of the tool names defined)
Came across after getting errors implementing tokens used code
Worked by selecting agent type of STRUCTURED_CHAT_ZERO_SHOT_REACT_DESCRIPTION
agent = initialize_agent(tools,
llm,
agent=AgentType.STRUCTURED_CHAT_ZERO_SHOT_REACT_DESCRIPTION,
verbose=True)
I have the same problem, but is not always. Sometimes the tools works fine and other times I got this error. It happens more frequently with GPT-3.5-turbo that with text-davinci-003. I don't have access to gpt-4 :(
How can it be solved?
Has anyone here tried Bard or Llama instead? I am on the waiting list for GPT-4, which certainly will improve this in relation to understanding which tool to use, however, I can't wait. It is a tricky bug because it works nicely sometimes.
I keep getting the same issue, even when using chat-zero-shot-react-description as agent instead of zero-shot-react-description. There should be an alternative way to handle this
same here, error details:
> Entering new AgentExecutor chain...
Answer:
{
"action": "Search",
"action\_input": "LangChain"
}
llama_print_timings: load time = 725.25 ms
llama_print_timings: sample time = 17.24 ms / 31 runs ( 0.56 ms per token)
llama_print_timings: prompt eval time = 27869.05 ms / 316 tokens ( 88.19 ms per token)
llama_print_timings: eval time = 3580.91 ms / 30 runs ( 119.36 ms per token)
llama_print_timings: total time = 33527.57 ms
---------------------------------------------------------------------------
IndexError Traceback (most recent call last)
File ~/venv/lib/python3.11/site-packages/langchain/agents/chat/output_parser.py:21, in ChatOutputParser.parse(self, text)
20 try:
---> 21 action = text.split("```")[1]
22 response = json.loads(action.strip())
IndexError: list index out of range
During handling of the above exception, another exception occurred:
OutputParserException Traceback (most recent call last)
Cell In[5], line 5
3 tools = load_tools(tool_names)
4 agent = initialize_agent(tools, llm, agent="chat-zero-shot-react-description", verbose=True)
----> 5 agent.run("What is LangChain?")
File ~/venv/lib/python3.11/site-packages/langchain/chains/base.py:236, in Chain.run(self, callbacks, *args, **kwargs)
234 if len(args) != 1:
235 raise ValueError("`run` supports only one positional argument.")
--> 236 return self(args[0], callbacks=callbacks)[self.output_keys[0]]
238 if kwargs and not args:
239 return self(kwargs, callbacks=callbacks)[self.output_keys[0]]
File ~/venv/lib/python3.11/site-packages/langchain/chains/base.py:140, in Chain.__call__(self, inputs, return_only_outputs, callbacks)
138 except (KeyboardInterrupt, Exception) as e:
139 run_manager.on_chain_error(e)
--> 140 raise e
141 run_manager.on_chain_end(outputs)
142 return self.prep_outputs(inputs, outputs, return_only_outputs)
File ~/venv/lib/python3.11/site-packages/langchain/chains/base.py:134, in Chain.__call__(self, inputs, return_only_outputs, callbacks)
128 run_manager = callback_manager.on_chain_start(
129 {"name": self.__class__.__name__},
130 inputs,
131 )
132 try:
133 outputs = (
--> 134 self._call(inputs, run_manager=run_manager)
135 if new_arg_supported
136 else self._call(inputs)
137 )
138 except (KeyboardInterrupt, Exception) as e:
139 run_manager.on_chain_error(e)
File ~/venv/lib/python3.11/site-packages/langchain/agents/agent.py:953, in AgentExecutor._call(self, inputs, run_manager)
951 # We now enter the agent loop (until it returns something).
952 while self._should_continue(iterations, time_elapsed):
--> 953 next_step_output = self._take_next_step(
954 name_to_tool_map,
955 color_mapping,
956 inputs,
957 intermediate_steps,
958 run_manager=run_manager,
959 )
960 if isinstance(next_step_output, AgentFinish):
961 return self._return(
962 next_step_output, intermediate_steps, run_manager=run_manager
963 )
File ~/venv/lib/python3.11/site-packages/langchain/agents/agent.py:773, in AgentExecutor._take_next_step(self, name_to_tool_map, color_mapping, inputs, intermediate_steps, run_manager)
771 raise_error = False
772 if raise_error:
--> 773 raise e
774 text = str(e)
775 if isinstance(self.handle_parsing_errors, bool):
File ~/venv/lib/python3.11/site-packages/langchain/agents/agent.py:762, in AgentExecutor._take_next_step(self, name_to_tool_map, color_mapping, inputs, intermediate_steps, run_manager)
756 """Take a single step in the thought-action-observation loop.
757
758 Override this to take control of how the agent makes and acts on choices.
759 """
760 try:
761 # Call the LLM to see what to do.
--> 762 output = self.agent.plan(
763 intermediate_steps,
764 callbacks=run_manager.get_child() if run_manager else None,
765 **inputs,
766 )
767 except OutputParserException as e:
768 if isinstance(self.handle_parsing_errors, bool):
File ~/venv/lib/python3.11/site-packages/langchain/agents/agent.py:444, in Agent.plan(self, intermediate_steps, callbacks, **kwargs)
442 full_inputs = self.get_full_inputs(intermediate_steps, **kwargs)
443 full_output = self.llm_chain.predict(callbacks=callbacks, **full_inputs)
--> 444 return self.output_parser.parse(full_output)
File ~/venv/lib/python3.11/site-packages/langchain/agents/chat/output_parser.py:26, in ChatOutputParser.parse(self, text)
23 return AgentAction(response["action"], response["action_input"], text)
25 except Exception:
---> 26 raise OutputParserException(f"Could not parse LLM output: {text}")
OutputParserException: Could not parse LLM output: Answer:
{
"action": "Search",
"action\_input": "LangChain"
}
Has anyone here tried Bard or Llama instead? I am on the waiting list for GPT-4, which certainly will improve this in relation to understanding which tool to use, however, I can't wait. It is a tricky bug because it works nicely sometimes.
yeah, sometimes works, sometimes fail, any update?
The error is being caused by ChatOpenAI. Please make the following substitution:
Replace:
from langchain.chat_models import ChatOpenAI
llm = ChatOpenAI(temperature=0, model_name='gpt-3.5-turbo')
With:
from langchain.llms import OpenAI
llm = OpenAI(model_name='text-davinci-003', temperature=0)
Hello all, I am also facing the same issue of is not a valid tool, the agent executions try using different tools and ultimately stops throwing an error of iteration or time limit. Any sugestions would be appreciated to help on this issue
The error is being caused by ChatOpenAI. Please make the following substitution:
Replace:
from langchain.chat_models import ChatOpenAI
llm = ChatOpenAI(temperature=0, model_name='gpt-3.5-turbo')
With:
from langchain.llms import OpenAI
llm = OpenAI(model_name='text-davinci-003', temperature=0)
Same here
Using text-davinci-003
can solve this issue temporarily, but text-davinci-003
is 10x expensive than gpt-3.5-turbo
I am getting the same of @catskytw, any news on that?
same here with create_pandas_dataframe_agent
no matter if I use ChatOpenAI or OpenAI.
So, I was using the Google Search tool with LangChain and was facing this same issue. And here's what I understood and did the following to fix the error:
Before I explain everything here's my code:
import os
import dotenv
from langchain.llms import Cohere
from langchain.agents import initialize_agent, load_tools, AgentType
from langchain.utilities import GoogleSearchAPIWrapper
from langchain.tools import Tool
from langchain.prompts import PromptTemplate
from langchain.chains import LLMChain
if __name__ == "__main__":
dotenv.load_dotenv()
llm = Cohere(model="command-xlarge-20221108", cohere_api_key=os.environ["COHERE_API_KEY"])
prompt = PromptTemplate(
input_variables=["query"],
template="Write a summary of the following text: {query}"
)
summarize_chain = LLMChain(llm=llm, prompt=prompt)
search = GoogleSearchAPIWrapper()
search_tool = Tool(
name="LangChainSear",
description="Search Google for recent results.",
func=search.run,
)
summary_tool = Tool(
name="Summarizer",
description="Useful for summarizing texts.",
func=summarize_chain.run
)
agent = initialize_agent(
tools=[search_tool, summary_tool],
llm=llm,
agent=AgentType.ZERO_SHOT_REACT_DESCRIPTION,
verbose=True,
max_iterations=4
)
agent.run("What is LangChain? Then please summarize the results.")
First we need to make sure that in the description=
parameter of the Tool(...)
module we explicitly write what this tool would do. Why? because when we use the AgentType.ZERO_SHOT_REACT_DESCRIPTION
the Agent uses the ReAct framework to decide which tool to use based only on the tool's description.
Second, when we create our CSE on google, the name that we use while creating the CSE we use that same name in the Tool(...)
module for the Google Search API. For example, in my case I used LangChainSear
If we maintain the above cases, then the framework seems to work and produces correct results.
Let me know your thoughts.
Same issue here when using create_pandas_dataframe_agent
+ ChatOpenAI llm. The error disappears when I try with OpenAI, but I need to keep using gpt-3.5-turbo
.
So, I was using the Google Search tool with LangChain and was facing this same issue. And here's what I understood and did the following to fix the error:
Before I explain everything here's my code:
import os import dotenv from langchain.llms import Cohere from langchain.agents import initialize_agent, load_tools, AgentType from langchain.utilities import GoogleSearchAPIWrapper from langchain.tools import Tool from langchain.prompts import PromptTemplate from langchain.chains import LLMChain if __name__ == "__main__": dotenv.load_dotenv() llm = Cohere(model="command-xlarge-20221108", cohere_api_key=os.environ["COHERE_API_KEY"]) prompt = PromptTemplate( input_variables=["query"], template="Write a summary of the following text: {query}" ) summarize_chain = LLMChain(llm=llm, prompt=prompt) search = GoogleSearchAPIWrapper() search_tool = Tool( name="LangChainSear", description="Search Google for recent results.", func=search.run, ) summary_tool = Tool( name="Summarizer", description="Useful for summarizing texts.", func=summarize_chain.run ) agent = initialize_agent( tools=[search_tool, summary_tool], llm=llm, agent=AgentType.ZERO_SHOT_REACT_DESCRIPTION, verbose=True, max_iterations=4 ) agent.run("What is LangChain? Then please summarize the results.")
- First we need to make sure that in the
description=
parameter of theTool(...)
module we explicitly write what this tool would do. Why? because when we use theAgentType.ZERO_SHOT_REACT_DESCRIPTION
the Agent uses the ReAct framework to decide which tool to use based only on the tool's description.- Second, when we create our CSE on google, the name that we use while creating the CSE we use that same name in the
Tool(...)
module for the Google Search API. For example, in my case I usedLangChainSear
If we maintain the above cases, then the framework seems to work and produces correct results.
Let me know your thoughts.
Hi I did the same approach(building the tool chain/agent by myself instead of using create_panda_agent() but the result is pretty unstable with gpt-3.5 and ZERO_SHOT_REACT_DESCRIPTION parameter.
I found someone providing another work around by creating a blind director, but it's just reducing the fee of davinci slightly. https://www.youtube.com/watch?v=72KrhiRj68E
Due to using azure openai which still doesn't provide fuction calling, I have no chance to change the agent type from ZERO_SHOT_REACT_DESCRIPTION fo Function Calling(Fuction calling should solve this issue as far as I know)
Appreciate if anyone can give me some suggestions or tips. (:bow)
Me & @StereoPT have found a pretty good solution
With GPT-35-Turbo LLM (we were using the Azure Open AI but strongly believe this also works with the regular one as should be the same model), we wanted to mess around with a CSV so we used this: https://python.langchain.com/docs/modules/agents/toolkits/pandas
Basically we were getting an error everytime gpt generated code because... it was wrapped in ` All we added was: prefix
agent = create_pandas_dataframe_agent(
prefix="Remove any ` from the Action Input",
llm=self.model,
df=df,
verbose=True,
max_iterations=5,
)
We never had any problems after this simple change, hope this helps!
Thank you for your reply @ricardosantosduarte , but it didn't work for me. The only strategy that works for me is to use davinci as my OpenAI model, which is really expensive compared to GPT-3.5-turbo.
Thanks @ricardosantosduarte , but it doesn't work on my side.
watching this issue
I have a similar issue, but using the HuggingFacePipeline decorator on the model tiiuae/falcon-40b. The model always gets confused with action and action input:
When running this:
agent.run("Which host wrote the last message according to the timestamps?")
I get this:
Thought: I should use the sort_values
method to sort the dataframe by the @timestamp
column.
Action: df.sort_values(by='@timestamp')
Action Input: df
Observation: df.sort_values(by='@timestamp')
is not a valid tool, try another one.
In other words, it thinks the "Action" is is the "Action Input", although the action should be "Action: python_repl_ast" I got confused myself when reading this, maybe a better prompt would involve different namings? Maybe using Tool: instead of Action?
Of course, some part of it might be the limitations of falcon-40b, but the model is quite capable (see huggingfaces leaderboard) and it writes the correct query.
Thought: I think there will be a lot of output, but I think it is best to use the timestamp to see if there is a difference. Action: use python_repl_ast to see if there is a difference Action Input: "host.timestamp" ... Observation: use python_repl_ast to see if there is a difference is not a valid tool, try another one. Thought: I am still trying to find a valid tool to use, but I can see there is a timestamp difference between the two hosts Action: use python_repl_ast to see if there is a difference Action Input: "host.timestamp" Observation: use python_repl_ast to see if there is a difference is not a valid tool, try another one.
I was having the same issue as many above with the pandas dataframe agent. After playing around with different models/agent_types I found that the following configuration seems to work ok (I have no idea why):
from langchain.agents import create_pandas_dataframe_agent
from langchain.agents.agent_types import AgentType
from langchain.chat_models import ChatOpenAI
agent = create_pandas_dataframe_agent(
ChatOpenAI(temperature=0, model='gpt-3.5-turbo-0613'),
df,
verbose=True,
agent_type=AgentType.OPENAI_FUNCTIONS,
agent_executor_kwargs={"handle_parsing_errors":True}
)```
I was having the same issue as many above with the pandas dataframe agent. After playing around with different models/agent_types I found that the following configuration seems to work ok (I have no idea why):
from langchain.agents import create_pandas_dataframe_agent from langchain.agents.agent_types import AgentType from langchain.chat_models import ChatOpenAI agent = create_pandas_dataframe_agent( ChatOpenAI(temperature=0, model='gpt-3.5-turbo-0613'), df, verbose=True, agent_type=AgentType.OPENAI_FUNCTIONS, agent_executor_kwargs={"handle_parsing_errors":True} )```
It's the AgentType.OPENAI_FUNCTIONS that made the difference
watching this issue
Watching this issue
this is maddening
I think I found a workaround for this issue which requires a bit of prompt engineering
In your prompt, you should mention how the model would answer
Question: Write a Python
agent.run("""
For instance:
Question: Find out how much 2 plus 2 is.
Thought: I must use the Python shell to calculate 2 + 2
Action: Python REPL
Action Input:
2 + 2
Observation: 4
Thought: I now know the answer
Final Answer: 4
Example 2:
Question: You have a variable age in your scope. If it's greater or equal than 21, say OK. Else, say Nay.
Thought: I should write an if/else block in the Python shell.
Action: Python REPL
Action Input:
if age >= 21:
print("OK") # this line has four spaces at the beginning
else:
print("Nay") # this line has four spaces at the beginning
Observation: OK
Thought: I now know the answer
Final Answer: I have executed the task successfully.
Now begin for real!
Question: Write a Python script that prints "Hello, world!"
""")
This worked for me
Source: https://betterprogramming.pub/creating-my-first-ai-agent-with-vicuna-and-langchain-376ed77160e3
Another workaround is to change line 113 in
langchain/agents/openai_functions_agent/base.py
From
_tool_input = json.loads(function_call["arguments"])
To
json_python_code = json.dumps(function_call)
python_code = json.loads(json_python_code)
_tool_input = json.loads(python_code["arguments"])
I am facing an issue related to this https://github.com/langchain-ai/langchain/issues/8407
Could someone help?
Whole heartedly agree with the comment 'this is maddening'. Can't think of another time in my life when a bug was so randomly irreproducible. It works for hours and then shits the bed. While I certainly appreciate all the flavors of workarounds being tossed out here - is there anyone who has a theory as to what the actual issue is that is causing the agent to completely lose sight of a tool it references earlier in the observation?
Pure guess, but it almost feels like there's an (a)synchronous issue at play. Like maybe the library isn't quite finished being referenced/loaded before the agent begins to execute.
Pure guess, but it almost feels like there's an (a)synchronous issue at play. Like maybe the library isn't quite finished being referenced/loaded before the agent begins to execute.
Fantastic theory @TimLafferty . We will need someone smarter than me to validate that :)
@mvonoven For me it feels more like a confusing prompt that is generated by langchain agents. I always kept running into a similar chain thought.
In your case, the llm understands what it needs to do, but doesn't understand how to format it and the format confused me too as a human 🤖.
After "action:" langchain wants it to write a tool name, right? But imo, in the agent prompts, instead of writing action, it should be just written as "tool:" and "action input:" should just be "input:"
A tool name by itself is not an action. I think this has not been changed because most people are using good models such as openai's gpt's so they haven't run into this issue as much.
Some of the comments actually provided a hint for the answer. You should tell the agent in system prompt the definition of action with the set of available options.
Action: the action to take, this should be one of the following [Tool_A, Tool_B, Tool_C]
OR
Action: the action to take, this should be one of the provided tools names.
On run time you should see the following:
Action: Tool_A
Action Input: 'Foo Bar'
Observation: ...
Using create_pandas_dataframe_agent
is a bit of a nightmare has anyone successfully got it to work with GPT3.5 Turbo? They are multiple issues using create_pandas_dataframe_agent
with the existing default text-davinci-003
import pandas as pd
from typing import Any, Dict, List, Optional, Union
from langchain.agents.mrkl.output_parser import MRKLOutputParser
from langchain.callbacks import get_openai_callback
from langchain.callbacks.base import BaseCallbackHandler
from langchain.input import print_text
from langchain.llms import OpenAI
from langchain.memory import (
ConversationKGMemory,
ConversationSummaryMemory,
ConversationBufferWindowMemory,
CombinedMemory)
from langchain.agents import (
create_pandas_dataframe_agent,
AgentExecutor,
AgentType
)
from langchain.schema import AgentAction, AgentFinish, LLMResult
class CSVCallbackHandler(BaseCallbackHandler):
"""Callback Handler that prints to std out."""
def __init__(self, color: Optional[str] = None) -> None:
"""Initialize callback handler."""
self.color = color
def on_llm_start(self, serialized: Dict[str, Any], prompts: List[str], **kwargs: Any) -> None:
"""Print out the prompts."""
print("on_llm_start")
def on_llm_end(self, response: LLMResult, **kwargs: Any) -> None:
"""Do nothing."""
print("on_llm_end")
print(response)
def on_llm_new_token(self, token: str, **kwargs: Any) -> None:
"""Do nothing."""
print("______on_llm_new_token______")
print(token)
def on_llm_error(self, error: Union[Exception, KeyboardInterrupt], **kwargs: Any) -> None:
"""Do nothing."""
pass
def on_chain_start(self, serialized: Dict[str, Any], inputs: Dict[str, Any], **kwargs: Any) -> None:
"""Print out that we are entering a chain."""
class_name = serialized.get("name", "")
print("on_chain_start")
def on_chain_end(self, outputs: Dict[str, Any], **kwargs: Any) -> None:
print(outputs)
def on_chain_error(self, error: Union[Exception, KeyboardInterrupt], **kwargs: Any) -> None:
pass
def on_tool_start(self, serialized: Dict[str, Any], input_str: str, **kwargs: Any,) -> None:
pass
def on_agent_action(self, action: AgentAction, color: Optional[str] = None, **kwargs: Any) -> Any:
print(action)
def on_tool_end(self, output: str, color: Optional[str] = None, observation_prefix: Optional[str] = "Observation: ", llm_prefix: Optional[str] = None, **kwargs: Any,) -> None:
if observation_prefix is not None:
print(f"\n{observation_prefix}", output)
print(output)
if llm_prefix is not None:
print_text(f"\n{llm_prefix}", output)
print(output)
def on_tool_error(self, error: Union[Exception, KeyboardInterrupt], **kwargs: Any) -> None:
pass
def on_text(self, text: str, color: Optional[str] = None, end: str = "", **kwargs: Any,) -> None:
pass
def on_agent_finish(self, finish: AgentFinish, color: Optional[str] = None, **kwargs: Any) -> None:
pass
DEBUG = True
csv_cb = CSVCallbackHandler()
llm_code = OpenAI(temperature=0, callbacks=[csv_cb], streaming=False) # gpt-3.5-turbo-16k-0613
chat_history_buffer = ConversationBufferWindowMemory(k=50, memory_key="chat_history_buffer", input_key="input")
chat_history_summary = ConversationSummaryMemory(llm=llm_code, memory_key="chat_history_summary", input_key="input")
chat_history_summary.chat_memory.add_ai_message(
"This is data is extracted from multiple csv files and data is about campaigns"
)
chat_history_KG = ConversationKGMemory(llm=llm_code, memory_key="chat_history_KG", input_key="input")
memory = CombinedMemory(memories=[chat_history_buffer, chat_history_summary, chat_history_KG])
MULTI_DF_PREFIX = """
You are working with {num_dfs} pandas dataframes in Python named df1, df2, etc which represent multiple excel and csv files.
Your task is to analyze the df data and answer questions related to it.
You should use the tools below, summary and conversation history to answer the question:
If you do not have an answer to the question do not make one up reply I apologize I dont know the answer to the question is not related to the data:
Summary of the whole conversation:
{chat_history_summary}
Last few messages between you and user:
{chat_history_buffer}
Entities that the conversation is about:
{chat_history_KG}
"""
csv_data_1 = "/docs/91271562-2c89-4708-a861-67ebab915b26/Deco2u-Campaigns-20-May-202020-Jun-2023.csv"
csv_data_2 = "/docs/91271562-2c89-4708-a861-67ebab915b26/Untitled-report-Jun-3-2020-to-Jul-3-2023.csv"
csv_files = [csv_data_1, csv_data_2]
df = []
for item in csv_files:
if not isinstance(item, str):
raise ValueError(f"Expected str, got {type(item)}")
df.append(pd.read_csv(item))
pd_agent: AgentExecutor = create_pandas_dataframe_agent(
llm_code,
df,
agent_type=AgentType.ZERO_SHOT_REACT_DESCRIPTION,
verbose=DEBUG,
prefix=MULTI_DF_PREFIX,
agent_executor_kwargs={
"memory": memory,
"handle_parsing_errors": True,
"output_parser": MRKLOutputParser()
},
input_variables=['num_dfs', 'dfs_head', 'input',
'agent_scratchpad', 'chat_history_buffer',
'chat_history_summary', 'chat_history_KG']
)
if DEBUG:
print("^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^")
print(pd_agent.agent.llm_chain.prompt.template)
print("^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^")
with get_openai_callback() as cb:
while True:
query = input("Ask me anything...")
resp = pd_agent.run({'input': query,
'agent_scratchpad': "", 'chat_history_buffer': chat_history_buffer,
'chat_history_summary': chat_history_summary, 'chat_history_KG': chat_history_KG})
print(resp)
not sure if this is the cause...currently custom tools cannot be loaded by their names using the "load_tools" function. If that function is ever called under the hood to load the custom tools, it won't work. See: https://github.com/langchain-ai/langchain/issues/5774
I have had this problem with a tool I never subclassed and simply instantiated as Tool(...)
but with a long name (4 words) that when shortened down to two words fixed the issue. Odd behavior if you ask me but it might help someone here too.
Hi, @gcsun,
I'm helping the LangChain team manage their backlog and am marking this issue as stale. From what I understand, the issue you reported involves the second agent causing all tools to become invalid, displaying an error message indicating that the tool is not valid. There have been discussions around potential workarounds, such as using a different agent type or changing the tool's name, as well as suggestions to modify the system prompt and potential issues related to custom tools and tool names. However, the issue remains unresolved.
Could you please confirm if this issue is still relevant to the latest version of the LangChain repository? If it is, please let the LangChain team know by commenting on the issue. Otherwise, feel free to close the issue yourself, or the issue will be automatically closed in 7 days.
Thank you for your understanding and cooperation.
I got it working by using
chat-zero-shot-react-description
as agent instead ofzero-shot-react-description
As far as i know, that's not documented anywhere.
from langchain.chat_models import ChatOpenAI from langchain.agents import load_tools, initialize_agent llm = ChatOpenAI(temperature=0) tools = load_tools(["python_repl"]) agent= initialize_agent(tools, llm, agent="chat-zero-shot-react-description", verbose=True) agent.run("Can i print a number in python?")
Thanks sooooo much - that resolved it.
initialize_agent is deprecated, any new?
I encapsulated an agent into a tool ,and load it into another agent 。 when the second agent run, all the tools become invalid. After calling any tool, the output is “*** is not a valid tool, try another one.
why! agent cannot be a tool?