crewAIInc / crewAI

Framework for orchestrating role-playing, autonomous AI agents. By fostering collaborative intelligence, CrewAI empowers agents to work together seamlessly, tackling complex tasks.
https://crewai.com
MIT License
18.53k stars 2.54k forks source link

Tool callbacks #747

Open BespalovSergey opened 2 months ago

BespalovSergey commented 2 months ago

Hello, we use callback processing to monitor the operation of tools, tell why you use the private _run method when calling tools, as a result, tool callbacks are not called. It is quite important to monitor and process the operation of tools with callbacks.

theCyberTech commented 2 months ago

Private functions / methods in Python are not enforced, meaning you can still call them, call back functions do work in crewAI, feel free to drop you crewAI code here's

whimo commented 2 months ago

BaseTool class in Langchain has a public run method, which is the way a tool should be called. Besides calling self._run, it also takes care of things such as error handling, and more importantly, fires callbacks on tool start and end.

We use callbacks for monitoring and debugging agents, but currently tool calls are not logged because of callbacks not being called. What do you think about either using BaseTool's public methods or firing the events inside CrewAi code if the former is not possible for some reason?

theCyberTech commented 2 months ago

I understand but I don't think there is a functional difference when defining run or _run, correct me if I'm wrong

whimo commented 2 months ago

Not sure what you mean by that. The difference is in the auxiliary stuff that happens inside run before and after it executes _run, and it's quite important for our use cases.

You can refer to the Langchain code: https://github.com/langchain-ai/langchain/blob/27b9ea14a51398e8ff2a4972026178778f85134b/libs/core/langchain_core/tools.py#L355

This code calls on_tool_start, on_tool_error, on_tool_end callbacks regardless of the logic of the _run method.

github-actions[bot] commented 6 days ago

This issue is stale because it has been open for 30 days with no activity. Remove stale label or comment or this will be closed in 5 days.

whimo commented 6 days ago

Bumping this because the issue is still present.