Closed ssg-kstewart closed 3 weeks ago
For additional context, it appears as though the tool calls are not occurring successfully as the response body results in a IndexError here. Messages with the response body injected appear as shown:
[
{
"role": "user",
"content": "REDACTED"
},
{
"role": "assistant",
"content": "REDACTED"
},
{
"role": "user",
"content": "RESPONSEBODY: {'id': 'msg_bdrk_018qmbt8JrxDnLUhEY7XbCtd', 'type': 'message', 'role': 'assistant', 'model': 'claude-3-sonnet-20240229', 'content': [], 'stop_reason': 'end_turn', 'stop_sequence': None, 'usage': {'input_tokens': 531, 'output_tokens': 3}}"
},
{
"role": "assistant",
"content": "REDACTED"
},
{
"role": "user",
"content": "RESPONSEBODY: {'id': 'msg_bdrk_01Ww79eCr8Bz1ZsSyrkgbSUH', 'type': 'message', 'role': 'assistant', 'model': 'claude-3-sonnet-20240229', 'content': [], 'stop_reason': 'end_turn', 'stop_sequence': None, 'usage': {'input_tokens': 796, 'output_tokens': 3}}"
}
]
EDIT: Note, this is after forcing an alternating message pattern using the following logic. Without alternating, I run into the same issue requiring the alternation. Ultimately, it looks as though tools are not actually being called successfully.
for i in range(1, len(messages)):
if messages[i]['role'] == messages[i - 1]['role']:
messages[i]['role'] = 'assistant' if messages[i - 1]['role'] == 'user' else 'user'
Attached is additional debug information including a sorted diff of two tool calls, one from the GPT-4o model and one from the Bedrock Anthropic implementation. The structure differences appear to be related to this PR which I am currently working from.
bedrock-tool-call-sorted.json gpt-tool-call-sorted.json sorted-diff.json
{ {
"generations": [ "generations": [
[ [
{ {
"generation_info": null, | "generation_info": {
> "finish_reason": "tool_calls",
> "logprobs": null
> },
"message": { "message": {
"id": [ "id": [
"langchain", "langchain",
"schema", "schema",
"messages", "messages",
"AIMessage" "AIMessage"
], ],
"kwargs": { "kwargs": {
"additional_kwargs": { "additional_kwargs": {
"model_id": "anthropic.claude-3-sonnet-20240229 <
"stop_reason": "tool_use", <
"tool_calls": [ "tool_calls": [
{ {
"function": { "function": {
"arguments": { | "arguments": "{\"entity_name\":\"Person\"
"entity_name": "Person", | "name": "test_calc_wrapper"
"expression": "firstName != null" | },
}, | "id": "call_eszDySpx0r99Rq9OVuupxHiF",
"name": "test_calc_wrapper", | "type": "function"
"type": "function" <
} <
} }
], | ]
"usage": { <
"completion_tokens": 148, <
"prompt_tokens": 362, <
"total_tokens": 510 <
} <
}, },
"content": "REDACTED", | "content": "",
"id": "run-67445d61-86f4-49cf-9de7-031f35af3d5b-0 | "id": "run-58754c7e-8b91-4b37-9b7e-cabca937413c-0
"invalid_tool_calls": [], "invalid_tool_calls": [],
"response_metadata": { "response_metadata": {
"model_id": "anthropic.claude-3-sonnet-20240229 | "finish_reason": "tool_calls",
"stop_reason": "tool_use", | "logprobs": null,
"tool_calls": [ | "model_name": "gpt-4o",
{ | "system_fingerprint": "fp_319be4768e",
"function": { | "token_usage": {
"arguments": { | "completion_tokens": 23,
"entity_name": "Person", | "prompt_tokens": 14037,
"expression": "firstName != null" | "total_tokens": 14060
}, <
"name": "test_calc_wrapper", <
"type": "function" <
} <
} <
], <
"usage": { <
"completion_tokens": 148, <
"prompt_tokens": 362, <
"total_tokens": 510 <
} }
}, },
"tool_calls": [], | "tool_calls": [
"type": "ai" | {
> "args": {
> "entity_name": "Person",
> "expression": "firstName != null"
> },
> "id": "call_eszDySpx0r99Rq9OVuupxHiF",
> "name": "test_calc_wrapper"
> }
> ],
> "type": "ai",
> "usage_metadata": {
> "input_tokens": 14037,
> "output_tokens": 23,
> "total_tokens": 14060
> }
}, },
"lc": 1, "lc": 1,
"type": "constructor" "type": "constructor"
}, },
"text": "REDACTED", | "text": "",
"type": "ChatGeneration" "type": "ChatGeneration"
} }
] ]
], ],
"llm_output": { "llm_output": {
"model_id": "anthropic.claude-3-sonnet-20240229-v1:0", | "model_name": "gpt-4o",
"stop_reason": "tool_use", | "system_fingerprint": "fp_319be4768e",
"tool_calls": [ | "token_usage": {
{ | "completion_tokens": 23,
"function": { | "prompt_tokens": 14037,
"arguments": { | "total_tokens": 14060
"entity_name": "Person", <
"expression": "firstName != null" <
}, <
"name": "test_calc_wrapper", <
"type": "function" <
} <
} <
], <
"usage": { <
"completion_tokens": 148, <
"prompt_tokens": 362, <
"total_tokens": 510 <
} }
}, },
"run": null "run": null
} }
I have resolved my issue via a PR. For addtional details, refer to this comment.
I am attempting to use ChatBedrock with an Anthropic Claude Sonnet model (anthropic.claude-3-sonnet-20240229-v1:0) in an implementation of LangGraph's collaborative multi-agent system. I am receiving an error as shown in a below stack trace where there is an expectation that roles must alternate. Messages passed in to
_prepare_input_and_invoke
are structured as so (with the message content redacted):I assume this is expected behavior in regards to the multi-agent system and see similar message structure when using ChatOpenAI in place of ChatBedrock where there are multiple messages of type "ai" in succession.