[FEATURE] Register Agent Workflow Step

[owaiskazi19]

Is your feature request related to a problem?

To register an agent through Flow Framework, we need to have an associated workflow step.

Register agent API looks like the below:

{
    "name": "Test_Agent_For_CoT",
    "type": "cot",
    "description": "this is a test agent",
    "llm": {
        "model_id": "ldzS04kBxRPZ5cnWrqpd",
        "parameters": {
            "max_iteration": 5,
            "stop_when_no_tool_found": true
        }
    },
    "memory": {
        "type": "conversation_buffer_window"
    },
    "tools": [
        {
            "name": "MLModelTool",
            "alias": "language_model_tool",
            "description": "A general tool to answer any question. But it can't handle math problem.",
            "parameters": {
                "model_id": "ldzS04kBxRPZ5cnWrqpd",
                "prompt": "Answer the question as best you can.\nBegin!\n\nHuman: ${parameters.questoin}\nAI:",
                "response_filter": "$.choices[0].message.content"
            }
        },
        {
            "name": "MathTool",
            "description": "A general tool to calculate any math problem. The action input must be valid math expression, like 2+3, (20-1)*2",
            "parameters": {
            }
        },
        {
            "name": "VectorDBTool",
            "description": "A tool to search opensearch index with natural language quesiotn. Action Input: <natrual language question>",
            "parameters": {
                "model_id": "2LDDzYkBy7u7wFRWX51f",
                "index": "my_test_data",
                "embedding_field": "embedding",
                "source_field": "text",
                "input": "${parameters.question}"
            }
        }
    ]
}

To explain the above API further:

1. Every tool has a modelID associated with it.
2. Every sub agent can have any number of tools associated with it.
3. A root agent can have multiple sub agents.

What solution would you like?

We can separate it out to 2 different workflow steps:

1. ToolStep
2. AgentStep

From the ToolStep we can store the object of the tool i.e. MLToolSpec in the WorkflowData map with the key as tools as it's a list of WorkflowData. Next in the RegisterAgentStep when we encounter the tools field, we can look for the key tools in the WorkflowData and add it to the tools list. Let's call it as mlToolSpecList and we an pass the same to MLAgentBuilder to call the register Agent API later. This way we can have a a sub agent to have a list of tools.

But for the tools the model_id is the part of parameters

           "name": "MLModelTool",
            "alias": "language_model_tool",
            "description": "A general tool to answer any question. But it can't handle math problem.",
            "parameters": {
                "model_id": "ldzS04kBxRPZ5cnWrqpd",
                "prompt": "Answer the question as best you can.\nBegin!\n\nHuman: ${parameters.questoin}\nAI:",
                "response_filter": "$.choices[0].message.content"
            }

1. How can we add model_id from the previous step of deploy_model under parameters here?

What alternatives have you considered?

Another solution would be, to have a separate ToolStep for every tool. For example, MathToolStep for a MathTool.

Do you have any additional context?

From the RegisterAgentStep we would get an agent_id for every sub agent and the same response for the root agent.

2. How can we differentiate the agentIds and store the response in the status API?
3. Does storing different tools in WorkflowData with the key tools cause any override? Should we store the tool as tool-<type-of-tool> as the key?

[joshpalis]

How can we add model_id from the previous step of deploy_model under parameters here?

From what I see, it would be necessary for us to build the parameters within the ToolStep, retrieving the model_id from the WorkflowData.

How can we differentiate the agentIds and store the response in the status API?

From what I understand about resources created, each resource will be associated with the workflow step ID that creates the resource.

Does storing different tools in WorkflowData with the key tools cause any override? Should we store the tool as tool- as the key?

Whenever a workflow step completes, it returns an object of WorkflowData which is appended to the overall List<WorkflowData>, so there shoudn't be any updates to existing entries within the WorkflowData.

[dbwiddis]

I don't think adding a new workflow step for every new Tool is scalable.

There is a Tool interface which defines all the inputs; they all have a name, alias, descriptions, and parameter map.

What more do we need to track for different tools?

[dbwiddis]

How can we add model_id from the previous step of deploy_model under parameters here?

I think this is really only relevant for the vector db tool, but can't we just throw in the placeholder ${{previous_step.model_id}} there?

[dbwiddis]

I think this is really only relevant for the vector db tool, but can't we just throw in the placeholder ${{previous_step.model_id}} there?

And for this we need the unique step id passed. Alter the ProcessNode call to Workflow steps to also include the id as a parameter.

[owaiskazi19]

Sample Template from Creating a connector to Registering an agent:

{
    "name": "tool-register-agent",
    "description": "test case",
    "use_case": "REGISTER_AGENT",
    "version": {
        "template": "1.0.0",
        "compatibility": [
            "2.12.0",
            "3.0.0"
        ]
    },
    "workflows": {
        "provision": {
            "nodes": [
                {
                    "id": "workflow_step_1",
                    "type": "create_connector",
                    "user_inputs": {
                        "name": "OpenAI Chat Connector",
                        "description": "The connector to public OpenAI model service for GPT 3.5",
                        "protocol": "http",
                        "version": "1",
                        "parameters": {
                            "endpoint": "api.openai.com",
                            "model": "flow-register-remote-test-gpt-3.5-turbo"
                        },
                        "credential": {
                            "openAI_key": "12345"
                        },
                        "actions": [
                            {
                                "action_type": "predict",
                                "method": "POST",
                                "url": "https://${parameters.endpoint}/v1/chat/completions"
                            }
                        ]
                    }
                },
                {
                    "id": "workflow_step_2",
                    "type": "register_remote_model",
                    "previous_node_inputs": {
                        "workflow_step_1": "connector_id"
                    },
                    "user_inputs": {
                        "name": "flow-register-remote-test-gpt-3.5-turbo",
                        "function_name": "remote",
                        "description": "test model"
                    }
                },
                {
                    "id": "workflow_step_3",
                    "type": "deploy_model",
                    "previous_node_inputs": {
                        "workflow_step_2": "model_id"
                    }
                },
                {
                    "id": "workflow_step_4",
                    "type": "create_tool",
                    "user_inputs": {
                        "name": "MLModelTool",
                        "type": "MLModelTool",
                        "alias": "language_model_tool",
                        "description": "A general tool to answer any question. But it can't handle math problem",
                        "parameters": {
                            "model_id": "workflow2.model_id",
                            "prompt": "Answer the question as best you can.",
                            "response_filter": "choices[0].message.content"
                        }
                    }
                },
                {
                    "id": "workflow_step_5",
                    "type": "create_tool",
                    "user_inputs": {
                        "name": "MathTool",
                        "type": "MathTool",
                        "description": "A general tool to calculate any math problem. The action input must be valid math expression, like 2+3",
                        "parameters": {
                            "max_iteration": 5
                        }
                    }
                },
                {
                    "id": "workflow_step_6",
                    "type": "register_agent",
                    "previous_node_inputs": {
                        "workflow_step_2": "model_id",
                        "workflow_step_4": "tools",
                        "workflow_step_5": "tools"
                    },
                    "user_inputs": {
                        "name": "DEMO-Test_Agent_For_CoT",
                        "type": "cot",
                        "description": "this is a test agent",
                        "parameters": {
                            "hello": "world"
                        },
                        "llm.parameters": {
                            "max_iteration": "5",
                            "stop_when_no_tool_found": "true"
                        },
                        "memory": {
                            "type": "conversation_buffer_window"
                        },
                        "app_type": "chatbot",
                        "created_time": 1689793598499,
                        "last_updated_time": 1689793598530
                    }
                }
            ],
            "edges": [
                {
                    "source": "workflow_step_1",
                    "dest": "workflow_step_2"
                },
                {
                    "source": "workflow_step_2",
                    "dest": "workflow_step_3"
                },
                {
                    "source": "workflow_step_2",
                    "dest": "workflow_step_6"
                },
                {
                    "source": "workflow_step_4",
                    "dest": "workflow_step_6"
                },
                {
                    "source": "workflow_step_5",
                    "dest": "workflow_step_6"
                }
            ]
        }
    }
}

Agent Registration Logs:

[2023-12-01T00:31:20,016][INFO ][o.o.f.w.RegisterRemoteModelStep] [ip-172-31-56-214] Remote Model registration successful
[2023-12-01T00:31:20,016][INFO ][o.o.f.w.ProcessNode      ] [ip-172-31-56-214] Finished workflow_step_2.
[2023-12-01T00:31:20,016][INFO ][o.o.f.w.ProcessNode      ] [ip-172-31-56-214] Starting workflow_step_3.
[2023-12-01T00:31:20,016][INFO ][o.o.f.w.ProcessNode      ] [ip-172-31-56-214] Starting workflow_step_6.
[2023-12-01T00:31:20,018][INFO ][o.o.f.w.RegisterAgentStep] [ip-172-31-56-214] MODEL ID FOR LLM ed_JIowBX7NSsKd4f9LC
[2023-12-01T00:31:20,022][INFO ][o.o.m.a.d.TransportDeployModelAction] [ip-172-31-56-214] Will deploy model on these nodes: OkZ4g5z7S_umhu691dXt4w
[2023-12-01T00:31:20,030][INFO ][o.o.f.w.RegisterAgentStep] [ip-172-31-56-214] Agent registration successful
[2023-12-01T00:31:20,030][INFO ][o.o.f.w.ProcessNode      ] [ip-172-31-56-214] Finished workflow_step_6.
[2023-12-01T00:31:20,053][INFO ][o.o.m.m.MLModelManager   ] [ip-172-31-56-214] Set connector d9_JIowBX7NSsKd4f9KK for the model: ed_JIowBX7NSsKd4f9LC
[2023-12-01T00:31:20,054][INFO ][o.o.f.w.DeployModelStep  ] [ip-172-31-56-214] Model deployment state COMPLETED
[2023-12-01T00:31:20,054][INFO ][o.o.f.w.ProcessNode      ] [ip-172-31-56-214] Finished workflow_step_3.
[2023-12-01T00:31:20,055][INFO ][o.o.f.t.ProvisionWorkflowTransportAction] [ip-172-31-56-214] Provisioning completed successfully for workflow dt_JIowBX7NSsKd4S9I6
[2023-12-01T00:31:20,071][INFO ][o.o.f.t.ProvisionWorkflowTransportAction] [ip-172-31-56-214] updated workflow dt_JIowBX7NSsKd4S9I6 state to COMPLETED
[2023-12-01T00:31:20,078][ERROR][o.o.f.w.CreateConnectorStep] [ip-172-31-56-214] Failed to update workflow state with newly created resource: {}
org.opensearch.transport.RemoteTransportException: [ip-172-31-56-214][127.0.0.1:9300][indices:data/write/update[s]]
Caused by: org.opensearch.index.engine.VersionConflictEngineException: [dt_JIowBX7NSsKd4S9I6]: version conflict, required seqNo [33], primary term [16]. current document has seqNo [34] and primary term [16]
[2023-12-01T00:31:20,097][INFO ][o.o.m.a.f.TransportForwardAction] [ip-172-31-56-214] deploy model done with state: DEPLOYED, model id: ed_JIowBX7NSsKd4f9LC
[2023-12-01T00:31:20,099][INFO ][o.o.m.a.d.TransportDeployModelOnNodeAction] [ip-172-31-56-214] deploy model task done et_JIowBX7NSsKd4f9LY
[2023-12-01T00:31:20,099][INFO ][o.o.m.m.MLModelManager   ] [ip-172-31-56-214] Completed setting connector d9_JIowBX7NSsKd4f9KK in the model ed_JIowBX7NSsKd4f9LC

[owaiskazi19]

Sample Template for Registering an agent:

{
    "name": "tool-register-agent",
    "description": "test case",
    "use_case": "REGISTER_AGENT",
    "version": {
        "template": "1.0.0",
        "compatibility": [
            "2.12.0",
            "3.0.0"
        ]
    },
    "workflows": {
        "provision": {
            "nodes": [
                {
                    "id": "workflow_step_1",
                    "type": "create_tool",
                    "user_inputs": {
                        "name": "MLModelTool",
                        "type": "MLModelTool",
                        "alias": "language_model_tool",
                        "description": "A general tool to answer any question. But it can't handle math problem",
                        "parameters": {
                            "model_id": "workflow2.model_id",
                            "prompt": "Answer the question as best you can.",
                            "response_filter": "choices[0].message.content"
                        }
                    }
                },
                {
                    "id": "workflow_step_2",
                    "type": "create_tool",
                    "user_inputs": {
                        "name": "MathTool",
                        "type": "MathTool",
                        "description": "A general tool to calculate any math problem. The action input must be valid math expression, like 2+3",
                        "parameters": {
                            "max_iteration": 5
                        }
                    }
                },
                {
                    "id": "workflow_step_3",
                    "type": "register_agent",
                    "previous_node_inputs": {
                        "workflow_step_1": "tools",
                        "workflow_step_2": "tools"
                    },
                    "user_inputs": {
                        "name": "DEMO-Test_Agent_For_CoT",
                        "type": "cot",
                        "description": "this is a test agent",
                        "parameters": {"hello": "world"},
                        "llm.model_id": "ldzS04kBxRPZ5cnWrqpd",
                        "llm.parameters": {
                                "max_iteration": "5",
                                "stop_when_no_tool_found": "true"
                        },

                        "memory": {
                            "type": "conversation_buffer_window"
                        },
                        "app_type": "chatbot",
                        "created_time": 1689793598499,
                        "last_updated_time": 1689793598530
                    }
                }
            ],
            "edges": [
                {
                    "source": "workflow_step_1",
                    "dest": "workflow_step_3"
                },
                {
                    "source": "workflow_step_2",
                    "dest": "workflow_step_3"
                }
            ]
        }
    }
}

Response:

[2023-11-30T23:22:36,420][INFO ][o.o.f.t.ProvisionWorkflowTransportAction] [ip-172-31-56-214] Queueing process [workflow_step_3]. Must wait for [workflow_step_1, workflow_step_2] to complete first.
[2023-11-30T23:22:36,421][INFO ][o.o.f.w.ToolStep         ] [ip-172-31-56-214] Tool registered successfully MathTool
[2023-11-30T23:22:36,421][INFO ][o.o.f.w.ToolStep         ] [ip-172-31-56-214] Tool registered successfully MLModelTool
[2023-11-30T23:22:36,421][INFO ][o.o.f.w.ProcessNode      ] [ip-172-31-56-214] Finished workflow_step_2.
[2023-11-30T23:22:36,421][INFO ][o.o.f.w.ProcessNode      ] [ip-172-31-56-214] Finished workflow_step_1.
[2023-11-30T23:22:36,421][INFO ][o.o.f.w.ProcessNode      ] [ip-172-31-56-214] Starting workflow_step_3.
[2023-11-30T23:22:36,439][INFO ][o.o.f.w.RegisterAgentStep] [ip-172-31-56-214] Remote Agent registration successful
[2023-11-30T23:22:36,440][INFO ][o.o.f.w.ProcessNode      ] [ip-172-31-56-214] Finished workflow_step_3.
[2023-11-30T23:22:36,440][INFO ][o.o.f.t.ProvisionWorkflowTransportAction] [ip-172-31-56-214] Provisioning completed successfully for workflow HfiKIowBRCoR2fQYgHWd
[2023-11-30T23:22:36,444][INFO ][o.o.f.t.ProvisionWorkflowTransportAction] [ip-172-31-56-214] updated workflow HfiKIowBRCoR2fQYgHWd state to PROVISIONING
[2023-11-30T23:22:36,454][INFO ][o.o.f.t.ProvisionWorkflowTransportAction] [ip-172-31-56-214] updated workflow HfiKIowBRCoR2fQYgHWd state to COMPLETED
[2023-11-30T23:22:37,239][INFO ][o.o.m.c.MLSyncUpCron     ] [ip-172-31-56-214] ML configuration already initialized, no action needed
[2023-12-01T00:07:13,190][INFO ][o.o.f.u.EncryptorUtils   ] [ip-172-31-56-214] Config has already been initialized

[owaiskazi19]

{
                    "id": "workflow_step_1",
                    "type": "create_tool",
                    "user_inputs": {
                        "name": "MLModelTool",
                        "type": "MLModelTool",
                        "alias": "language_model_tool",
                        "description": "A general tool to answer any question. But it can't handle math problem",
                        "parameters": {
                            "model_id": "ldzS04kBxRPZ5cnWrqpd",
                            "prompt": "Answer the question as best you can.",
                            "response_filter": "choices[0].message.content"
                        }
                    }
                }

Tools parameters with model_id already provided. Look for PARAMETERS MAP

2-01T01:09:22,412][INFO ][o.o.f.t.ProvisionWorkflowTransportAction] [ip-172-31-56-214] Queueing process [workflow_step_3]. Must wait for [workflow_step_1, workflow_step_2] to complete first.
[2023-12-01T01:09:22,412][INFO ][o.o.f.w.ProcessNode      ] [ip-172-31-56-214] Starting workflow_step_2.
[2023-12-01T01:09:22,413][INFO ][o.o.f.w.ToolStep         ] [ip-172-31-56-214] PARAMETERS MAP {response_filter=choices[0].message.content, model_id=ldzS04kBxRPZ5cnWrqpd, prompt=Answer the question as best you can.}
[2023-12-01T01:09:22,413][INFO ][o.o.f.w.ToolStep         ] [ip-172-31-56-214] PARAMETERS MAP {max_iteration=5}
[2023-12-01T01:09:22,414][INFO ][o.o.f.w.ToolStep         ] [ip-172-31-56-214] Tool registered successfully MathTool
[2023-12-01T01:09:22,414][INFO ][o.o.f.w.ToolStep         ] [ip-172-31-56-214] Tool registered successfully MLModelTool
[2023-12-01T01:09:22,414][INFO ][o.o.f.w.ProcessNode      ] [ip-172-31-56-214] Finished workflow_step_1.
[2023-12-01T01:09:22,414][INFO ][o.o.f.w.ProcessNode      ] [ip-172-31-56-214] Finished workflow_step_2.
[2023-12-01T01:09:22,414][INFO ][o.o.f.w.ProcessNode      ] [ip-172-31-56-214] Starting workflow_step_3.

[owaiskazi19]

{
                    "id": "workflow_step_4",
                    "type": "create_tool",
                    "previous_node_inputs": {
                        "workflow_step_2": "model_id"
                    },
                    "user_inputs": {
                        "name": "MLModelTool",
                        "type": "MLModelTool",
                        "alias": "language_model_tool",
                        "description": "A general tool to answer any question. But it can't handle math problem",
                        "parameters": {
                            "model_id": "workflow2.model_id",
                            "prompt": "Answer the question as best you can.",
                            "response_filter": "choices[0].message.content"
                        }
                    }
                },

Tools parameters without modelId and fetching it from previous steps. Look for PARAMETERS MAP FROM PREVIOUS NODE

[2023-12-01T01:10:55,310][INFO ][o.o.f.w.ProcessNode      ] [ip-172-31-56-214] Starting workflow_step_3.
[2023-12-01T01:10:55,310][INFO ][o.o.f.w.ToolStep         ] [ip-172-31-56-214] PARAMETERS MAP FROM PREVIOUS NODE {response_filter=choices[0].message.content, model_id=pTXtIowB4rvYAGS5vo1B, prompt=Answer the question as best you can.}
[2023-12-01T01:10:55,310][INFO ][o.o.f.w.ToolStep         ] [ip-172-31-56-214] Tool registered successfully MLModelTool
[2023-12-01T01:10:55,311][INFO ][o.o.f.w.ProcessNode      ] [ip-172-31-56-214] Finished workflow_step_4.
[2023-12-01T01:10:55,311][INFO ][o.o.f.w.ProcessNode      ] [ip-172-31-56-214] Starting workflow_step_6.
[2023-12-01T01:10:55,315][INFO ][o.o.m.a.d.TransportDeployModelAction] [ip-172-31-56-214] Will deploy model on these nodes: OkZ4g5z7S_umhu691dXt4w
[2023-12-01T01:10:55,342][INFO ][o.o.m.m.MLModelManager   ] [ip-172-31-56-214] Set connector ozXtIowB4rvYAGS5vo0O for the model: pTXtIowB4rvYAGS5vo1B
[2023-12-01T01:10:55,343][INFO ][o.o.f.w.DeployModelStep  ] [ip-172-31-56-214] Model deployment state COMPLETED
[2023-12-01T01:10:55,344][INFO ][o.o.f.w.ProcessNode      ] [ip-172-31-56-214] Finished workflow_step_3.

[owaiskazi19]

Root Agent Template (Look for AgentTool, it takes an agent_id):

{
    "name": "tool-register-agent",
    "description": "test case",
    "use_case": "REGISTER_AGENT",
    "version": {
        "template": "1.0.0",
        "compatibility": [
            "2.12.0",
            "3.0.0"
        ]
    },
    "workflows": {
        "provision": {
            "nodes": [
                {
                    "id": "workflow_step_1",
                    "type": "create_tool",
                    "user_inputs": {
                        "name": "MLModelTool",
                        "type": "MLModelTool",
                        "alias": "language_model_tool",
                        "description": "A general tool to answer any question. But it can't handle math problem",
                        "parameters": {
                            "model_id": "ldzS04kBxRPZ5cnWrqpd",
                            "prompt": "Answer the question as best you can.",
                            "response_filter": "choices[0].message.content"
                        }
                    }
                },
                {
                    "id": "workflow_step_2",
                    "type": "register_agent",
                    "previous_node_inputs": {
                        "workflow_step_1": "tools"
                    },
                    "user_inputs": {
                        "name": "TOOLS-TOOLS-TOOLS",
                        "type": "cot",
                        "description": "this is a test agent",
                        "parameters": {
                            "hello": "world"
                        },
                        "llm.model_id": "ldzS04kBxRPZ5cnWrqpd",
                        "llm.parameters": {
                            "max_iteration": "5",
                            "stop_when_no_tool_found": "true"
                        },
                        "memory": {
                            "type": "conversation_buffer_window"
                        },
                        "app_type": "chatbot",
                        "created_time": 1689793598499,
                        "last_updated_time": 1689793598530
                    }
                },
                {
                    "id": "workflow_step_3",
                    "type": "create_tool",
                    "previous_node_inputs": {
                        "workflow_step_2": "agent_id"
                    },
                    "user_inputs": {
                        "name": "AgentTool",
                        "type": "AgentTool",
                        "description": "Root Agent Tool",
                        "parameters": {
                            "max_iteration": 5
                        }
                    }
                },
                {
                    "id": "workflow_step_4",
                    "type": "register_agent",
                    "previous_node_inputs": {
                        "workflow_step_1": "tools",
                        "workflow_step_3": "tools"
                    },
                    "user_inputs": {
                        "name": "TOOLS-TOOLS-TOOLS",
                        "type": "cot",
                        "description": "this is a test agent",
                        "parameters": {
                            "hello": "world"
                        },
                        "llm.model_id": "ldzS04kBxRPZ5cnWrqpd",
                        "llm.parameters": {
                            "max_iteration": "5",
                            "stop_when_no_tool_found": "true"
                        },
                        "memory": {
                            "type": "conversation_buffer_window"
                        },
                        "app_type": "chatbot",
                        "created_time": 1689793598499,
                        "last_updated_time": 1689793598530
                    }
                }
            ],
            "edges": [
                {
                    "source": "workflow_step_1",
                    "dest": "workflow_step_2"
                },
                {
                    "source": "workflow_step_2",
                    "dest": "workflow_step_3"
                },
                {
                    "source": "workflow_step_1",
                    "dest": "workflow_step_4"
                },
                {
                    "source": "workflow_step_3",
                    "dest": "workflow_step_4"
                }
            ]
        }
    }
}

Response: Look for PARAMETERS LIST

irst.
[2023-12-02T22:11:25,653][INFO ][o.o.f.w.ToolStep         ] [ip-172-31-56-214] PARAMETERS LIST {response_filter=choices[0].message.content, model_id=ldzS04kBxRPZ5cnWrqpd, prompt=Answer the question as best you can.}
[2023-12-02T22:11:25,653][INFO ][o.o.f.w.ToolStep         ] [ip-172-31-56-214] Tool registered successfully MLModelTool
[2023-12-02T22:11:25,653][INFO ][o.o.f.w.ProcessNode      ] [ip-172-31-56-214] Finished workflow_step_1.
[2023-12-02T22:11:25,653][INFO ][o.o.f.w.ProcessNode      ] [ip-172-31-56-214] Starting workflow_step_2.
[2023-12-02T22:11:25,661][INFO ][o.o.f.w.RegisterAgentStep] [ip-172-31-56-214] Agent registration successful for the agent irKWLIwB9rygHQ58IepW
[2023-12-02T22:11:25,662][INFO ][o.o.f.w.ProcessNode      ] [ip-172-31-56-214] Finished workflow_step_2.
[2023-12-02T22:11:25,662][INFO ][o.o.f.w.ProcessNode      ] [ip-172-31-56-214] Starting workflow_step_3.
[2023-12-02T22:11:25,662][INFO ][o.o.f.w.ToolStep         ] [ip-172-31-56-214] PARAMETERS LIST {max_iteration=5, agent_id=irKWLIwB9rygHQ58IepW}
[2023-12-02T22:11:25,662][INFO ][o.o.f.w.ToolStep         ] [ip-172-31-56-214] Tool registered successfully AgentTool
[2023-12-02T22:11:25,663][INFO ][o.o.f.w.ProcessNode      ] [ip-172-31-56-214] Finished workflow_step_3.
[2023-12-02T22:11:25,663][INFO ][o.o.f.w.ProcessNode      ] [ip-172-31-56-214] Starting workflow_step_4.
[2023-12-02T22:11:25,664][INFO ][o.o.f.t.ProvisionWorkflowTransportAction] [ip-172-31-56-214] updated workflow ibKWLIwB9rygHQ58BOpz state to PROVISIONING
[2023-12-02T22:11:25,668][INFO ][o.o.f.w.RegisterAgentStep] [ip-172-31-56-214] Agent registration successful for the agent i7KWLIwB9rygHQ58Iepg
[2023-12-02T22:11:25,668][INFO ][o.o.f.w.ProcessNode      ] [ip-172-31-56-214] Finished workflow_step_4.
[2023-12-02T22:11:25,668][INFO ][o.o.f.t.ProvisionWorkflowTransportAction] [ip-172-31-56-214] Provisioning completed successfully for workflow ibKWLIwB9rygHQ58BOpz
[2023-12-02T22:11:25,680][INFO ][o.o.f.t.ProvisionWorkflowTransportAction] [ip-172-31-56-214] updated workflow ibKWLIwB9rygHQ58BOpz state to COMPLETED

[owaiskazi19]

Multiple tools are added for an agent:

[2023-12-03T03:12:12,889][INFO ][o.o.f.w.RegisterAgentStep] [ip-172-31-56-214] TOOL ADDED MathTool
[2023-12-03T03:12:12,889][INFO ][o.o.f.w.RegisterAgentStep] [ip-172-31-56-214] TOOL ADDED MLModelTool
[2023-12-03T03:12:12,889][INFO ][o.o.f.w.RegisterAgentStep] [ip-172-31-56-214] TOOLS LIST [org.opensearch.ml.common.agent.MLToolSpec@2f4dbb90, org.opensearch.ml.common.agent.MLToolSpec@635894e4]

[dbwiddis]

TOOL ADDED MathTool TOOL ADDED MLModelTool

With Math and ML Models, there's nothing we can't do!!!!

[owaiskazi19]

Complete template:

{
    "name": "tool-register-agent",
    "description": "test case",
    "use_case": "REGISTER_AGENT",
    "version": {
        "template": "1.0.0",
        "compatibility": [
            "2.12.0",
            "3.0.0"
        ]
    },
    "workflows": {
        "provision": {
            "nodes": [
                {
                    "id": "workflow_step_1",
                    "type": "create_connector",
                    "user_inputs": {
                        "name": "OpenAI Chat Connector",
                        "description": "The connector to public OpenAI model service for GPT 3.5",
                        "protocol": "http",
                        "version": "1",
                        "parameters": {
                            "endpoint": "api.openai.com",
                            "model": "flow-register-remote-test-gpt-3.5-turbo"
                        },
                        "credential": {
                            "openAI_key": "12345"
                        },
                        "actions": [
                            {
                                "action_type": "predict",
                                "method": "POST",
                                "url": "https://${parameters.endpoint}/v1/chat/completions"
                            }
                        ]
                    }
                },
                {
                    "id": "workflow_step_2",
                    "type": "register_remote_model",
                    "previous_node_inputs": {
                        "workflow_step_1": "connector_id"
                    },
                    "user_inputs": {
                        "name": "flow-register-remote-test-gpt-3.5-turbo",
                        "function_name": "remote",
                        "description": "test model"
                    }
                },
                {
                    "id": "workflow_step_3",
                    "type": "deploy_model",
                    "previous_node_inputs": {
                        "workflow_step_2": "model_id"
                    }
                },
                {
                    "id": "workflow_step_4",
                    "type": "create_tool",
                    "previous_node_inputs": {
                        "workflow_step_3": "model_id"
                    },
                    "user_inputs": {
                        "name": "MLModelTool",
                        "type": "MLModelTool",
                        "alias": "language_model_tool",
                        "description": "A general tool to answer any question. But it can't handle math problem",
                        "parameters": {
                            "prompt": "Answer the question as best you can.",
                            "response_filter": "choices[0].message.content"
                        }
                    }
                },
                {
                    "id": "workflow_step_5",
                    "type": "create_tool",
                    "user_inputs": {
                        "name": "MathTool",
                        "type": "MathTool",
                        "description": "A general tool to calculate any math problem. The action input must be valid math expression, like 2+3",
                        "parameters": {
                            "max_iteration": 5
                        }
                    }
                },
                {
                    "id": "workflow_step_6",
                    "type": "register_agent",
                    "previous_node_inputs": {
                        "workflow_step_5": "tools"
                    },
                    "user_inputs": {
                        "name": "Sub Agent",
                        "type": "cot",
                        "description": "this is a test agent",
                        "parameters": {
                            "hello": "world"
                        },
                        "llm.model_id": "ldzS04kBxRPZ5cnWrqpd",
                        "llm.parameters": {
                            "max_iteration": "5",
                            "stop_when_no_tool_found": "true"
                        },
                        "memory": {
                            "type": "conversation_buffer_window"
                        },
                        "app_type": "chatbot",
                        "created_time": 1689793598499,
                        "last_updated_time": 1689793598530
                    }
                },
                {
                    "id": "workflow_step_7",
                    "type": "create_tool",
                    "previous_node_inputs": {
                        "workflow_step_6": "agent_id"
                    },
                    "user_inputs": {
                        "name": "AgentTool",
                        "type": "AgentTool",
                        "description": "Root Agent Tool",
                        "parameters": {
                            "max_iteration": 5
                        }
                    }
                },
                {
                    "id": "workflow_step_8",
                    "type": "register_agent",
                    "previous_node_inputs": {
                        "workflow_step_3": "model_id",
                        "workflow_step_4": "tools",
                        "workflow_step_7": "tools"
                    },
                    "user_inputs": {
                        "name": "DEMO-Test_Agent_For_CoT",
                        "type": "cot",
                        "description": "this is a test agent",
                        "parameters": {
                            "hello": "world"
                        },
                        "llm.parameters": {
                            "max_iteration": "5",
                            "stop_when_no_tool_found": "true"
                        },
                        "memory": {
                            "type": "conversation_buffer_window"
                        },
                        "app_type": "chatbot",
                        "created_time": 1689793598499,
                        "last_updated_time": 1689793598530
                    }
                }
            ],
            "edges": [
                {
                    "source": "workflow_step_1",
                    "dest": "workflow_step_2"
                },
                {
                    "source": "workflow_step_2",
                    "dest": "workflow_step_3"
                },
                {
                    "source": "workflow_step_3",
                    "dest": "workflow_step_8"
                },
                {
                    "source": "workflow_step_3",
                    "dest": "workflow_step_4"
                },
                {
                    "source": "workflow_step_4",
                    "dest": "workflow_step_8"
                },
                {
                    "source": "workflow_step_5",
                    "dest": "workflow_step_6"
                },
                {
                    "source": "workflow_step_6",
                    "dest": "workflow_step_7"
                },
                {
                    "source": "workflow_step_7",
                    "dest": "workflow_step_8"
                }
            ]
        }
    }
}

[dbwiddis]

In YAML

---
name: tool-register-agent
description: test case
use_case: REGISTER_AGENT
version:
  template: 1.0.0
  compatibility:
  - 2.12.0
  - 3.0.0
workflows:
  provision:
    nodes:
    - id: workflow_step_1
      type: create_connector
      user_inputs:
        name: OpenAI Chat Connector
        description: The connector to public OpenAI model service for GPT 3.5
        protocol: http
        version: '1'
        parameters:
          endpoint: api.openai.com
          model: flow-register-remote-test-gpt-3.5-turbo
        credential:
          openAI_key: '12345'
        actions:
        - action_type: predict
          method: POST
          url: https://${parameters.endpoint}/v1/chat/completions
    - id: workflow_step_2
      type: register_remote_model
      previous_node_inputs:
        workflow_step_1: connector_id
      user_inputs:
        name: flow-register-remote-test-gpt-3.5-turbo
        function_name: remote
        description: test model
    - id: workflow_step_3
      type: deploy_model
      previous_node_inputs:
        workflow_step_2: model_id
    - id: workflow_step_4
      type: create_tool
      previous_node_inputs:
        workflow_step_3: model_id
      user_inputs:
        name: MLModelTool
        type: MLModelTool
        alias: language_model_tool
        description: A general tool to answer any question. But it can't handle math
          problem
        parameters:
          prompt: Answer the question as best you can.
          response_filter: choices[0].message.content
    - id: workflow_step_5
      type: create_tool
      user_inputs:
        name: MathTool
        type: MathTool
        description: A general tool to calculate any math problem. The action input
          must be valid math expression, like 2+3
        parameters:
          max_iteration: 5
    - id: workflow_step_6
      type: register_agent
      previous_node_inputs:
        workflow_step_5: tools
      user_inputs:
        name: Sub Agent
        type: cot
        description: this is a test agent
        parameters:
          hello: world
        llm.model_id: ldzS04kBxRPZ5cnWrqpd
        llm.parameters:
          max_iteration: '5'
          stop_when_no_tool_found: 'true'
        memory:
          type: conversation_buffer_window
        app_type: chatbot
        created_time: 1689793598499
        last_updated_time: 1689793598530
    - id: workflow_step_7
      type: create_tool
      previous_node_inputs:
        workflow_step_7: agent_id
      user_inputs:
        name: AgentTool
        type: AgentTool
        description: Root Agent Tool
        parameters:
          max_iteration: 5
    - id: workflow_step_8
      type: register_agent
      previous_node_inputs:
        workflow_step_3: model_id
        workflow_step_4: tools
        workflow_step_7: tools
      user_inputs:
        name: DEMO-Test_Agent_For_CoT
        type: cot
        description: this is a test agent
        parameters:
          hello: world
        llm.parameters:
          max_iteration: '5'
          stop_when_no_tool_found: 'true'
        memory:
          type: conversation_buffer_window
        app_type: chatbot
        created_time: 1689793598499
        last_updated_time: 1689793598530
    edges:
    - source: workflow_step_1
      dest: workflow_step_2
    - source: workflow_step_2
      dest: workflow_step_3
    - source: workflow_step_3
      dest: workflow_step_8
    - source: workflow_step_3
      dest: workflow_step_4
    - source: workflow_step_4
      dest: workflow_step_8
    - source: workflow_step_5
      dest: workflow_step_6
    - source: workflow_step_6
      dest: workflow_step_7
    - source: workflow_step_7
      dest: workflow_step_8

[owaiskazi19]

Tested the above template using ml-commons APIs of Get Agent: GET _plugins/_ml/agents/i1jTNowBknzE5_26V-cO Sub Agent Response:

{
    "name": "Sub Agent",
    "type": "cot",
    "description": "this is a test agent",
    "llm": {
        "model_id": "ldzS04kBxRPZ5cnWrqpd",
        "parameters": {
            "max_iteration": "5",
            "stop_when_no_tool_found": "true"
        }
    },
    "tools": [
        {
            "type": "MathTool",
            "name": "MathTool",
            "description": "A general tool to calculate any math problem. The action input must be valid math expression, like 2+3",
            "parameters": {
                "max_iteration": "5"
            },
            "include_output_in_agent_response": false
        }
    ],
    "parameters": {
        "hello": "world"
    },
    "memory": {
        "type": "conversation_buffer_window"
    },
    "created_time": 1701726869262,
    "last_updated_time": 1701726869262,
    "app_type": "chatbot"
}

Root Agent Response:

{
    "name": "FLOW-FRAMEWORK-ROOT-AGENT",
    "type": "cot",
    "description": "this is a test agent",
    "llm": {
        "model_id": "jljTNowBknzE5_26V-c8",
        "parameters": {
            "max_iteration": "5",
            "stop_when_no_tool_found": "true"
        }
    },
    "tools": [
        {
            "type": "AgentTool",
            "name": "AgentTool",
            "description": "Root Agent Tool",
            "parameters": {
                "max_iteration": "5"
            },
            "include_output_in_agent_response": false
        },
        {
            "type": "MLModelTool",
            "name": "MLModelTool",
            "description": "A general tool to answer any question. But it can't handle math problem",
            "parameters": {
                "response_filter": "choices[0].message.content",
                "model_id": "jljTNowBknzE5_26V-c8",
                "prompt": "Answer the question as best you can."
            },
            "include_output_in_agent_response": false
        }
    ],
    "parameters": {
        "hello": "world"
    },
    "memory": {
        "type": "conversation_buffer_window"
    },
    "created_time": 1701726874365,
    "last_updated_time": 1701726874365,
    "app_type": "chatbot"
}

[amitgalitz]

Taking Owais' template and renaming steps as they might be used by users:

{
    "name": "tool-register-agent",
    "description": "test case",
    "use_case": "REGISTER_AGENT",
    "version": {
        "template": "1.0.0",
        "compatibility": [
            "2.12.0",
            "3.0.0"
        ]
    },
    "workflows": {
        "provision": {
            "nodes": [
                {
                    "id": "openAI_connector",
                    "type": "create_connector",
                    "user_inputs": {
                        "name": "OpenAI Chat Connector",
                        "description": "The connector to public OpenAI model service for GPT 3.5",
                        "protocol": "http",
                        "version": "1",
                        "parameters": {
                            "endpoint": "api.openai.com",
                            "model": "flow-register-remote-test-gpt-3.5-turbo"
                        },
                        "credential": {
                            "openAI_key": "12345"
                        },
                        "actions": [
                            {
                                "action_type": "predict",
                                "method": "POST",
                                "url": "https://${parameters.endpoint}/v1/chat/completions"
                            }
                        ]
                    }
                },
                {
                    "id": "gpt-3.5-model",
                    "type": "register_remote_model",
                    "previous_node_inputs": {
                        "openAI_connector": "connector_id"
                    },
                    "user_inputs": {
                        "name": "flow-register-remote-test-gpt-3.5-turbo",
                        "function_name": "remote",
                        "description": "test model"
                    }
                },
                {
                    "id": "deployed-gpt-3.5-model",
                    "type": "deploy_model",
                    "previous_node_inputs": {
                        "gpt-3.5-model": "model_id"
                    }
                },
                {
                    "id": "ml_model_tool",
                    "type": "create_tool",
                    "previous_node_inputs": {
                        "deployed-gpt-3.5-model": "model_id"
                    },
                    "user_inputs": {
                        "name": "MLModelTool",
                        "type": "MLModelTool",
                        "alias": "language_model_tool",
                        "description": "A general tool to answer any question. But it can't handle math problem",
                        "parameters": {
                            "prompt": "Answer the question as best you can.",
                            "response_filter": "choices[0].message.content"
                        }
                    }
                },
                {
                    "id": "math_tool",
                    "type": "create_tool",
                    "user_inputs": {
                        "name": "MathTool",
                        "type": "MathTool",
                        "description": "A general tool to calculate any math problem. The action input must be valid math expression, like 2+3",
                        "parameters": {
                            "max_iteration": 5
                        }
                    }
                },
                {
                    "id": "sub_agent",
                    "type": "register_agent",
                    "previous_node_inputs": {
                        "math_tool": "tools"
                    },
                    "user_inputs": {
                        "name": "Sub Agent",
                        "type": "cot",
                        "description": "this is a test agent",
                        "parameters": {
                            "hello": "world"
                        },
                        "llm.model_id": "ldzS04kBxRPZ5cnWrqpd",
                        "llm.parameters": {
                            "max_iteration": "5",
                            "stop_when_no_tool_found": "true"
                        },
                        "memory": {
                            "type": "conversation_index"
                        },
                        "app_type": "chatbot",
                        "created_time": 1689793598499,
                        "last_updated_time": 1689793598530
                    }
                },
                {
                    "id": "agent_tool",
                    "type": "create_tool",
                    "previous_node_inputs": {
                        "sub_agent": "agent_id"
                    },
                    "user_inputs": {
                        "name": "AgentTool",
                        "type": "AgentTool",
                        "description": "Root Agent Tool",
                        "parameters": {
                            "max_iteration": 5
                        }
                    }
                },
                {
                    "id": "root_agent",
                    "type": "register_agent",
                    "previous_node_inputs": {
                        "deployed-gpt-3.5-model": "model_id",
                        "ml_model_tool": "tools",
                        "agent_tool": "tools"
                    },
                    "user_inputs": {
                        "name": "DEMO-Test_Agent_For_CoT",
                        "type": "cot",
                        "description": "this is a test agent",
                        "parameters": {
                            "hello": "world"
                        },
                        "llm.parameters": {
                            "max_iteration": "5",
                            "stop_when_no_tool_found": "true"
                        },
                        "memory": {
                            "type": "conversation_index"
                        },
                        "app_type": "chatbot",
                        "created_time": 1689793598499,
                        "last_updated_time": 1689793598530
                    }
                }
            ],
            "edges": [
                {
                    "source": "openAI_connector",
                    "dest": "gpt-3.5-model"
                },
                {
                    "source": "gpt-3.5-model",
                    "dest": "deployed-gpt-3.5-model"
                },
                {
                    "source": "deployed-gpt-3.5-model",
                    "dest": "root_agent"
                },
                {
                    "source": "deployed-gpt-3.5-model",
                    "dest": "ml_model_tool"
                },
                {
                    "source": "ml_model_tool",
                    "dest": "root_agent"
                },
                {
                    "source": "math_tool",
                    "dest": "sub_agent"
                },
                {
                    "source": "sub_agent",
                    "dest": "agent_tool"
                },
                {
                    "source": "agent_tool",
                    "dest": "root_agent"
                }
            ]
        }
    }
}