This is a large PR that restructures the entire orchestration backend without touching the user-facing objects.
Pipeline:
a flow or task is run
a Controller object is created. The Controller is given a few parameters (the tasks to complete, the agents assigned, a flow) and its job is to orchestrate resources to achieve those goals
On each invocation, the Controller does the following:
looks for any tasks in the flow that are upstream of its assigned tasks and are ready to run
creates a pool of candidate agents assigned to those tasks
looks to see if one of the candidates is already selected as the active agent
if not, selects a new active agent
creates an AgentContext consisting of the agent itself, the agent's assigned tasks, and a sequence of events relevant to those tasks
creates tools for the agent based on the active tasks
compiles the events history into a series of LLM messages
runs the agent's LLM on those messages
calls any tools
determines if a new agent has been selected
The key to this backend is that all activity is recorded as a sequence of events that can optionally be persisted. These events represent everything that's happened in the workflow. When it's time to run an agent, we compile the events into LLM messages appropriate for that agent. For example, that agent's own responses are given the assistant role, and other agents' responses are given the user role with a system preamble explaining who the speaker is. Anecdotally this is far superior to the current approach in which all agents share the assistant role.
In addition, the compilation only includes events that are relevant to the agent's current objective. This means that if an agent is selected to work on task A, and in the next invocation a different agent is selected to work on task B, and A and B are completely independent, then the second agent will not be shown any of the first agent's responses, because they have no bearing on its work.
In essence, we compile a specialized view of the workflow universe for every agent invocation.
This is a large PR that restructures the entire orchestration backend without touching the user-facing objects.
Pipeline:
Controllerobject is created. The Controller is given a few parameters (the tasks to complete, the agents assigned, a flow) and its job is to orchestrate resources to achieve those goalsreadyto runAgentContextconsisting of the agent itself, the agent's assigned tasks, and a sequence of events relevant to those tasksThe key to this backend is that all activity is recorded as a sequence of events that can optionally be persisted. These events represent everything that's happened in the workflow. When it's time to run an agent, we compile the events into LLM messages appropriate for that agent. For example, that agent's own responses are given the
assistantrole, and other agents' responses are given theuserrole with asystempreamble explaining who the speaker is. Anecdotally this is far superior to the current approach in which all agents share theassistantrole.In addition, the compilation only includes events that are relevant to the agent's current objective. This means that if an agent is selected to work on task A, and in the next invocation a different agent is selected to work on task B, and A and B are completely independent, then the second agent will not be shown any of the first agent's responses, because they have no bearing on its work.
In essence, we compile a specialized view of the workflow universe for every agent invocation.