search

run-llama / llama_deploy

Deploy your agentic worfklows to production
https://docs.llamaindex.ai/en/stable/module_guides/llama_deploy/
MIT License
1.79k stars 184 forks source link

llama-agents :: text to sql :: infinite loop in terminal #190

Closed yb-sid closed 2 months ago

yb-sid commented 2 months ago

Hello , I'm trying to use llama-agents to solve text-to-sql problem for my work. I'm using azure openai's gpt-4o model with llama-index

I've created tools , each of these tool's function return string output : 

table_desc_tool = FunctionTool.from_defaults(
    fn=get_all_table_desc, # no arguments 
    name="get_all_table_desc",
    description="tool used to fetch descriptions of all tables present in schema"
)

column_desc_tool = FunctionTool.from_defaults(
    fn = get_column_desc, # one argument i,e table name
    name = "get_column_desc",
    description="tool used to fetch description for all the columns for a specified table"
)

sql_execute_tool = FunctionTool.from_defaults(
    fn = execute_generated_sql, # 1 argument , generated SQL
    name = "execute_generated_sql",
    description="tool used to execute generated SQL. Note: parameter must be a valid SQL select query"
)

defined following agents :

table_agent = FunctionCallingAgentWorker.from_tools(
    tools = [table_desc_tool],llm=omni_llm
).as_agent()
column_agent = FunctionCallingAgentWorker.from_tools(
    tools = [column_desc_tool] , llm = omni_llm
).as_agent()

execute_agent = FunctionCallingAgentWorker.from_tools(
    tools=[sql_execute_tool],llm= omni_llm
).as_agent()

master_agent = FunctionCallingAgentWorker.from_tools(llm=omni_llm).as_agent()

Created following services:

orchestrator_llm = get_llm("gpt-4o")

message_queue = SimpleMessageQueue(port = 8001)

control_plane = ControlPlaneServer(
    message_queue=message_queue,
    orchestrator=AgentOrchestrator(llm = orchestrator_llm,),
    port = 8002
)

table_agent_service = AgentService(
    agent = table_agent,
    message_queue=message_queue,
    description="Service to find all the tables and their description present in database",
    service_name="table_desc_service",
    port = 8003
)

column_agent_service = AgentService(
    agent = column_agent,
    message_queue=message_queue,
    description = "Service to find all the columns and it's descripition for a given table",
    service_name = "column_desc_service",
    port = 8004
)

execute_agent_service = AgentService(
    agent = execute_agent,
    message_queue=message_queue,
    description = "Useful to execute generated SQL and fetch data",
    service_name="sql_execute_service",
    port = 8005
)

generation_agent_service = AgentService(
    agent = master_agent,
    message_queue=message_queue,
    description = "Service used to generate SQL for a business question",
    service_name = "sql_generation_service",
    port = 8006
)

launched services using : 

import nest_asyncio

nest_asyncio.apply()

launcher = LocalLauncher(
    services=[table_agent_service,column_agent_service,
              execute_agent_service,generation_agent_service],
    control_plane=control_plane,
    message_queue=message_queue,
)

result = launcher.launch_single("find total spend in 2023")

print(f"Result = {result}")

The code launches, terminal shows services are registered : 

INFO:llama_agents.message_queues.simple - Consumer AgentService-e032bf0d-876f-49d4-8d50-6f8671430b03: table_desc_service has been registered.
INFO:llama_agents.message_queues.simple - Consumer AgentService-ef047209-244f-48c8-bcd6-28bd22536832: column_desc_service has been registered.
INFO:llama_agents.message_queues.simple - Consumer AgentService-07922165-5a23-4368-a5bb-ca97089c71fa: sql_execute_service has been registered.
INFO:llama_agents.message_queues.simple - Consumer AgentService-cb71e6fa-2641-496e-b3c9-abd17148a5a2: sql_generation_service has been registered.
INFO:llama_agents.message_queues.simple - Consumer d07aba13-ba14-465f-a8cd-0fe7bf1fc7eb: human has been registered.
INFO:llama_agents.message_queues.simple - Consumer ControlPlaneServer-5656c29a-304d-4f93-9297-390f4eba99f3: control_plane has been registered.
INFO:llama_agents.services.agent - table_desc_service launch_local
INFO:llama_agents.services.agent - column_desc_service launch_local
INFO:llama_agents.services.agent - sql_execute_service launch_local
INFO:llama_agents.services.agent - sql_generation_service launch_local
INFO:llama_agents.message_queues.base - Publishing message to 'control_plane' with action 'ActionTypes.NEW_TASK'
INFO:llama_agents.message_queues.simple - Launching message queue locally
INFO:llama_agents.services.agent - Processing initiated.
INFO:llama_agents.services.agent - Processing initiated.
INFO:llama_agents.services.agent - Processing initiated.
INFO:llama_agents.services.agent - Processing initiated.

But beyond that it goes into loop and does not work as expected. I was expecting following order:

As per below logs I can only see table_desc_service getting called multiple times. No other logs or agents are getting called , any way to debug this ? The logs keep on going into infinite and does not end. Also I don't get where the model is being used. No references / examples helped me out.

INFO:llama_agents.message_queues.base - Publishing message to 'control_plane' with action 'ActionTypes.NEW_TASK'
INFO:llama_agents.message_queues.simple - Launching message queue locally
INFO:llama_agents.services.agent - Processing initiated.
INFO:llama_agents.services.agent - Processing initiated.
INFO:llama_agents.services.agent - Processing initiated.
INFO:llama_agents.services.agent - Processing initiated.
INFO:llama_agents.message_queues.base - Publishing message to 'table_desc_service' with action 'ActionTypes.NEW_TASK'
INFO:llama_agents.message_queues.simple - Successfully published message 'control_plane' to consumer.
INFO:llama_agents.services.agent - Created new task: d98dea38-7e5a-4a0d-a8bc-814a90cb8269
INFO:llama_agents.message_queues.simple - Successfully published message 'table_desc_service' to consumer.
table descriptions fetched :  3
INFO:llama_agents.message_queues.base - Publishing message to 'control_plane' with action 'ActionTypes.COMPLETED_TASK'
INFO:llama_agents.message_queues.base - Publishing message to 'table_desc_service' with action 'ActionTypes.NEW_TASK'
INFO:llama_agents.message_queues.simple - Successfully published message 'control_plane' to consumer.
INFO:llama_agents.services.agent - Created new task: d98dea38-7e5a-4a0d-a8bc-814a90cb8269
INFO:llama_agents.message_queues.simple - Successfully published message 'table_desc_service' to consumer.
INFO:llama_agents.message_queues.base - Publishing message to 'control_plane' with action 'ActionTypes.COMPLETED_TASK'
INFO:llama_agents.message_queues.base - Publishing message to 'table_desc_service' with action 'ActionTypes.NEW_TASK'
INFO:llama_agents.message_queues.simple - Successfully published message 'control_plane' to consumer.
INFO:llama_agents.services.agent - Created new task: d98dea38-7e5a-4a0d-a8bc-814a90cb8269
INFO:llama_agents.message_queues.simple - Successfully published message 'table_desc_service' to consumer.
INFO:llama_agents.message_queues.base - Publishing message to 'control_plane' with action 'ActionTypes.COMPLETED_TASK'
INFO:llama_agents.message_queues.base - Publishing message to 'table_desc_service' with action 'ActionTypes.NEW_TASK'
INFO:llama_agents.message_queues.simple - Successfully published message 'control_plane' to consumer.
INFO:llama_agents.services.agent - Created new task: d98dea38-7e5a-4a0d-a8bc-814a90cb8269
INFO:llama_agents.message_queues.simple - Successfully published message 'table_desc_service' to consumer.
INFO:llama_agents.message_queues.base - Publishing message to 'control_plane' with action 'ActionTypes.COMPLETED_TASK'
INFO:llama_agents.message_queues.base - Publishing message to 'table_desc_service' with action 'ActionTypes.NEW_TASK'
INFO:llama_agents.message_queues.simple - Successfully published message 'control_plane' to consumer.
INFO:llama_agents.services.agent - Created new task: d98dea38-7e5a-4a0d-a8bc-814a90cb8269
INFO:llama_agents.message_queues.simple - Successfully published message 'table_desc_service' to consumer.
INFO:llama_agents.message_queues.base - Publishing message to 'control_plane' with action 'ActionTypes.COMPLETED_TASK'
INFO:llama_agents.message_queues.base - Publishing message to 'table_desc_service' with action 'ActionTypes.NEW_TASK'
INFO:llama_agents.message_queues.simple - Successfully published message 'control_plane' to consumer.
INFO:llama_agents.services.agent - Created new task: d98dea38-7e5a-4a0d-a8bc-814a90cb8269
INFO:llama_agents.message_queues.simple - Successfully published message 'table_desc_service' to consumer.
INFO:llama_agents.message_queues.base - Publishing message to 'control_plane' with action 'ActionTypes.COMPLETED_TASK'
INFO:llama_agents.message_queues.base - Publishing message to 'table_desc_service' with action 'ActionTypes.NEW_TASK'
INFO:llama_agents.message_queues.simple - Successfully published message 'control_plane' to consumer.
INFO:llama_agents.services.agent - Created new task: d98dea38-7e5a-4a0d-a8bc-814a90cb8269
INFO:llama_agents.message_queues.simple - Successfully published message 'table_desc_service' to consumer.
INFO:llama_agents.message_queues.base - Publishing message to 'control_plane' with action 'ActionTypes.COMPLETED_TASK'
INFO:llama_agents.message_queues.base - Publishing message to 'table_desc_service' with action 'ActionTypes.NEW_TASK'
INFO:llama_agents.message_queues.simple - Successfully published message 'control_plane' to consumer.
INFO:llama_agents.services.agent - Created new task: d98dea38-7e5a-4a0d-a8bc-814a90cb8269
INFO:llama_agents.message_queues.simple - Successfully published message 'table_desc_service' to consumer.
`table descriptions fetched :  3` # this is log from one of the tool which fetches table description
INFO:llama_agents.message_queues.base - Publishing message to 'control_plane' with action 'ActionTypes.COMPLETED_TASK'
INFO:llama_agents.message_queues.base - Publishing message to 'sql_generation_service' with action 'ActionTypes.NEW_TASK'
INFO:llama_agents.message_queues.simple - Successfully published message 'control_plane' to consumer.
INFO:llama_agents.services.agent - Created new task: d98dea38-7e5a-4a0d-a8bc-814a90cb8269
INFO:llama_agents.message_queues.simple - Successfully published message 'sql_generation_service' to consumer.
INFO:llama_agents.message_queues.base - Publishing message to 'control_plane' with action 'ActionTypes.COMPLETED_TASK'
INFO:llama_agents.message_queues.base - Publishing message to 'table_desc_service' with action 'ActionTypes.NEW_TASK'
INFO:llama_agents.message_queues.simple - Successfully published message 'control_plane' to consumer.
INFO:llama_agents.services.agent - Created new task: d98dea38-7e5a-4a0d-a8bc-814a90cb8269
INFO:llama_agents.message_queues.simple - Successfully published message 'table_desc_service' to consumer.
INFO:llama_agents.message_queues.base - Publishing message to 'control_plane' with action 'ActionTypes.COMPLETED_TASK'
INFO:llama_agents.message_queues.base - Publishing message to 'table_desc_service' with action 'ActionTypes.NEW_TASK'
INFO:llama_agents.message_queues.simple - Successfully published message 'control_plane' to consumer.
INFO:llama_agents.services.agent - Created new task: d98dea38-7e5a-4a0d-a8bc-814a90cb8269
INFO:llama_agents.message_queues.simple - Successfully published message 'table_desc_service' to consumer.
table descriptions fetched :  3
INFO:llama_agents.message_queues.base - Publishing message to 'control_plane' with action 'ActionTypes.COMPLETED_TASK'
INFO:llama_agents.message_queues.base - Publishing message to 'sql_generation_service' with action 'ActionTypes.NEW_TASK'

Any help to proceed or debug is appreciated.

I would also like to ask if this use case is valid for llama-agents ? I'm trying to validate other frameworks such as crewai and super

yb-sid commented 2 months ago

Any way I can inject prompts into control plane and orchestrator to guide it on how to get final results ?

logan-markewich commented 2 months ago

Imo I would build this with a workflow in llama-index, rather than llama-agents.

Working on pivoting llama-agents to be the place to deploy and scale workflows built in llama index

yb-sid commented 2 months ago

@logan-markewich , What is a workflow ? Do you mean create multiple-operational agents using core lib and performing orchestration by myself ?

I'm interested how the orchestration is being done by llama-agents . Why use a message queue like kafka / rabbitmq and how this queue is being used.

logan-markewich commented 2 months ago

Workflows: https://docs.llamaindex.ai/en/stable/module_guides/workflow/

Yea I mean writing workflows to orchestrate between agents yourself. It will be much more clear when things go wrong and how to debug them imo.

Message queues are used to pass messages between the control plane and services. Basically services are like job workers than consume jobs from the message queue.

The refactor is pivoting llama-agents to basically deploy workflows as services themselves, and help you scale that way. Its going to be very cool :)

yb-sid commented 2 months ago

thank you for the explanation @logan-markewich.

And keep up the great work!