Open jamesbraza opened 3 months ago
Here is a workaround:
from typing import cast
from unittest.mock import patch
from langchain.agents.openai_functions_agent.base import OpenAIFunctionsAgent
from langchain_core.agents import AgentAction, AgentFinish
from langchain_core.callbacks import BaseCallbackHandler, BaseCallbackManager, Callbacks
orig_plan = OpenAIFunctionsAgent.plan
...
def plan_with_injected_callbacks(
self,
intermediate_steps: list[tuple[AgentAction, str]],
callbacks: Callbacks = None,
**kwargs
) -> AgentAction | AgentFinish:
if self == agent_instance.agent:
# Work around https://github.com/langchain-ai/langchain/issues/22703
for callback in cast(list[BaseCallbackHandler], agent_instance.callbacks):
cast(BaseCallbackManager, callbacks).add_handler(callback, inherit=False)
return orig_plan(self, intermediate_steps, callbacks, **kwargs)
# NOTE: intentionally, I am not specifying the callback to invoke, as that
# would make the cost_callback be considered "inheritable" (which I don't want)
with patch.object(OpenAIFunctionsAgent, "plan", plan_with_injected_callbacks):
outputs = agent_instance.invoke(
input={"input": "Sleep a few times for 100-ms."},
# config=RunnableConfig(callbacks=[cost_callback]), # "Inheritable" callbacks
)
Checked other resources
Example Code
Error Message and Stack Trace (if applicable)
Traceback (most recent call last): File "/Users/user/code/repo/app/agents/a.py", line 28, in
assert cost_callback.total_cost > 0, "Agent planning should have been accounted for"
^^^^^^^^^^^^^^^^^^^^^^^^^^^^
AssertionError: Agent planning should have been accounted for
Description
LangChain has a useful concept of "inheritable" callbacks vs "local" callbacks, all managed by
CallbackManger
(source reference 1 and 2)invoke
invoke
Yesterday I discovered
AgentExecutor
does not use local callbacks for its planning step. I consider this a bug, as planning (e.gBaseSingleActionAgent.plan
) is a core behavior ofAgentExecutor
.The fix would be supporting
AgentExecutor
's local callbacks during planningSystem Info
langchain==0.2.3 langchain-community==0.2.4 langchain-core==0.2.5 langchain-openai==0.1.8