[Question]: Multi-agent capability - using agents as tools for other agents

[brycecf]

Question Validation

• [X] I have searched both the documentation and discord for an answer.

Question

Is there an existing interface that permits readily exposing agents as tools for other agents, thereby establishing a readily available multi-agent capability? Or, would it requiring defining a new ToolSpec?

FWIW, it appears AutoGen is poised to address this in the near future for LlamaIndex.

[dosubot[bot]]

Yes, the create_llama_agent and create_llama_chat_agent functions in LlamaIndex permit readily exposing agents as tools for other agents, thereby establishing a readily available multi-agent capability. These functions utilize the LlamaToolkit to initialize and load agent executors, which can be used to create and manage multiple agents. The create_llama_agent function allows for various agent types such as zero-shot-react-description, react-docstore, self-ask-with-search, and more, while the create_llama_chat_agent function specifically sets up a conversational agent using the CONVERSATIONAL_REACT_DESCRIPTION type.

Additionally, LlamaIndex offers a robust framework for defining and using tools within agents. Tools can be created using FunctionTool to wrap any user-defined function or QueryEngineTool to wrap existing query engines. These tools can be bundled into ToolSpecs for comprehensive service interfaces, such as Gmail, and utility tools like OnDemandLoaderTool and LoadAndSearchToolSpec to handle large data volumes. This framework allows for flexible and powerful multi-agent capabilities by enabling agents to use and expose a variety of tools seamlessly.

Moreover, the MultiDocumentAgentsPack class in LlamaIndex provides a structured way to build a multi-agent architecture over a set of documents. This class sets up individual document agents capable of QA and summarization and then establishes a top-level agent over these document agents. Each document agent is equipped with tools like QueryEngineTool for specific and holistic queries about the document. The top-level agent can use these document agents as tools, thereby creating a comprehensive multi-agent system. This setup allows for efficient handling of queries across multiple documents by leveraging specialized agents for each document ^{[1][2][3][4][5]}.

To continue talking to Dosu, mention @dosu.

[logan-markewich]

@brycecf maybe I'm missing something, but it should be as simple as wrapping an agent into a function tool?

Of course, there could always be a helper for this too, agent.as_tool() oe something, but setting up a function tool is also straightforward and easy to customize