Currently langchaingo does not support multi argument tools. Standards for LLM tool calling in foundational models are emerging hence we should adapt the library to utilise models fully. In addition AgentAction should support multi argument tool input.
The goal of this PR is to
[x] Review tool calling API's of several foundational model providers
[ ] Reach consensus on langchaingo multi argument tool call interfaces
[ ] Reach consensus on langchaingoagents tool calling interfaces
[ ] Implement changes to support multi argument tool calls
[ ] Implement changes to support agent multi argument tool calls
Review of Tool Call APIs
Most foundational models follows APIs which is this proposal fulfils.
### OpenAI
[Reference 1](https://platform.openai.com/docs/api-reference/runs/submitToolOutputs)
[Reference 2](https://platform.openai.com/docs/assistants/tools/function-calling/quickstart)
Request
```json
"tools": [
{
"type": "function",
"function": {
"name": "get_current_weather",
"description": "Get the current weather in a given location",
"parameters": {
"type": "object",
"properties": {
"location": {
"type": "string",
"description": "The city and state, e.g. San Francisco, CA"
},
"unit": {
"type": "string",
"enum": ["celsius", "fahrenheit"]
}
},
"required": ["location"]
}
}
}
],
```
Response
```json
"tool_calls": [
{
"id": "call_FthC9qRpsL5kBpwwyw6c7j4k",
"function": {
"arguments": "{\"location\": \"San Francisco, CA\"}",
"name": "get_rain_probability"
},
"type": "function"
},
{
"id": "call_RpEDoB8O0FTL9JoKTuCVFOyR",
"function": {
"arguments": "{\"location\": \"San Francisco, CA\", \"unit\": \"Fahrenheit\"}",
"name": "get_current_temperature"
},
"type": "function"
}
]
```
### Cohere
[Reference 1](https://docs.cohere.com/reference/chat)
[Reference 2](https://docs.cohere.com/docs/tool-use)
Identical to OpenAI up to field names and `arguments` is not string encoded.
### Mistal
[Reference 1](https://docs.mistral.ai/capabilities/function_calling/)
API is identical to OpenAI.
### Google Vertex
[Reference 1](https://cloud.google.com/vertex-ai/generative-ai/docs/multimodal/function-calling)
Identical to OpenAI up to field names.
type Tool struct {}
func (t Tool) Name() string {
return "GoogleSearch"
}
func (t Tool) Description() string {
return `
"A wrapper around Google Search. "
"Useful for when you need to answer questions about current events. "
"Always one of the first options when you need to find information on internet"
"Input should be a search query."`
}
func (t Tool) Call(ctx context.Context, input map[string]any) (string, error) {
query, ok := input["query"].(string)
if !ok {
return "", fmt.Errorf("could not obtain `query` in %v", input)
}
// tool stuff
}
func (t Tool) Schema() tools.Schema {
return tools.Schema{
Type: "object",
Properties: map[string]any{
"query": map[string]any{
"title": "query",
"type": "string",
},
},
Required: []string{"query"},
}
}
It's a job of the tool developer to define the schema and handling schema fields.
Proposal: updated AgentAction
For agents to support multi argument tools it's necessary to replace
[x] Name your Pull Request title clearly, concisely, and prefixed with the name of the primarily affected package you changed according to Good commit messages (such as memory: add interfaces for X, Y or util: add whizzbang helpers).
[x] Check that there isn't already a PR that solves the problem the same way to avoid creating a duplicate.
[ ] Provide a description in this PR that addresses what the PR is solving, or reference the issue that it solves (e.g. Fixes #123).
[ ] Describes the source of new concepts.
[ ] References existing implementations as appropriate.
Motivation
Currently
langchaingo
does not support multi argument tools. Standards for LLM tool calling in foundational models are emerging hence we should adapt the library to utilise models fully. In additionAgentAction
should support multi argument tool input.The goal of this PR is to
langchaingo
multi argument tool call interfaceslangchaingo
agents
tool calling interfacesReview of Tool Call APIs
Most foundational models follows APIs which is this proposal fulfils.
Proposal: updated
Tool
interfaceThis is a breaking change replacing previous
and introducing
Schema()
method.Example tool
It's a job of the tool developer to define the schema and handling schema fields.
Proposal: updated
AgentAction
For agents to support multi argument tools it's necessary to replace
Which replaces
ToolInput string
. This allows straightforward propagation for tool arguments from LLM toAgentAction
toTool
.Let me know what you think @tmc @eliben. I'm happy to implement and handle breaking changes.
PR Checklist
memory: add interfaces for X, Y
orutil: add whizzbang helpers
).Fixes #123
).golangci-lint
checks.