Model with no tools yields error after a successful tool call from another model: ValidationException: An error occurred (ValidationException) when calling the Converse operation: The toolConfig field must be defined when using toolUse and toolResult content blocks.

[DiogoPM9]

Requirements:

langchain==0.3.1
langchain-aws==0.2.7
langchain-chroma==0.1.4
langchain-community==0.3.1
langchain-core==0.3.15
langchain-openai==0.2.1
langchain-text-splitters==0.3.0
langgraph==0.2.28
langgraph-checkpoint==1.0.9
langsmith==0.1.129

The code of this issue is also present in #223. After the referenced issue was fixed and subsequently closed I attempted to execute the code in the issue and came across another error:

The assistant node successfully routes the query to the node that is powered by an LLM with tools. The tool call is successfully executed and the output added to state. For some reason, the assistant cannot handle the response of the tool call.

My thought was that perhaps the issue was being caused because the tool call output is wrapped in a ToolMessage object and the assistant node does not have access to any tools. However, the following attempts to resolve the issue did not work:

• Append the result as an AIMessage did not work since as it returned: ValidationException: An error occurred (ValidationException) when calling the Converse operation: The model that you are using requires the last turn in the conversation to be a user message. Add a user message to the conversation and try again.
• Append the result as a HumanMeessage did not work as it returned: ValidationException: An error occurred (ValidationException) when calling the Converse operation: messages.1.content: Conversation blocks and tool use blocks cannot be provided in the same turn.
• Append the result as a simple str object did not work as it returned: ValidationException: An error occurred (ValidationException) when calling the Converse operation: messages.1.content: Conversation blocks and tool use blocks cannot be provided in the same turn.

Code:

import os
import re
import pandas as pd
import json
from langgraph.prebuilt import ToolNode, tools_condition

from dotenv import load_dotenv
from langchain_chroma import Chroma
from langchain_community.embeddings import BedrockEmbeddings
from langchain_text_splitters import RecursiveCharacterTextSplitter
from langchain_community.document_loaders import PyPDFDirectoryLoader
from langchain_core.prompts import MessagesPlaceholder

from typing import Annotated, Literal
from typing_extensions import TypedDict

from IPython.display import Image, display

from langchain_aws import ChatBedrockConverse

from langchain_core.prompts import ChatPromptTemplate
from langchain_core.prompts import MessagesPlaceholder
from langchain_core.messages import AIMessage, HumanMessage
from langchain_core.messages.tool import ToolMessage

from langgraph.graph import StateGraph, START, END
from langgraph.graph.message import add_messages

from langchain_core.tools import tool
from langchain_community.document_loaders.athena import AthenaLoader
from langgraph.checkpoint.memory import MemorySaver

from pydantic import BaseModel

load_dotenv()

from langchain_openai import ChatOpenAI

@tool
def sales_records(
        start_date: str,
        end_date: str,
) -> dict:
    """
    Fetches the sales records

    Args:
        start_date: date to begin the search
        end_date: date to end the search
    Returns:
        a dictionary with the sales
    """
    return {"banana": 14, "apples": 20, "oranges": 2}

@tool
def context_retriever():
    """
    Retrieves the required context for the report.
    Returns:
        Context for the report.
    """
    return "The report must end with: And that is the way the news goes."

mistral = "mistral.mistral-large-2402-v1:0"
anthropic = "anthropic.claude-3-sonnet-20240229-v1:0"
# LLM = ChatBedrockConverse(
#         model=anthropic,
#         region_name=ZONE,
#         credentials_profile_name=PROFILE_NAME)

LLM = ChatOpenAI(
    model="gpt-4o",
    temperature=0,
    max_tokens=None,
    timeout=None,
    max_retries=2,
    api_key=API_KEY, 
)
class State(TypedDict):
    messages: Annotated[list, add_messages]
    feedback: str
    client_data: str
    context: str

members =["ToolCaller", "ReportMaker", "FeedbackRequester", "FINISH"]

# Architect
architect_instructions = f"""
You are a supervisor responsible for the delivery of a Sales report. Your workers comprise of {members}

The process to generate a report is as follows:

1. The ToolCaller will query the sales
2. The ToolCaller will retrieve the required context for the report
3. The Report Maker will generate a report
4. The FeedbackRequester asks the user for feedback
5. If there is feedback then call Report Maker again. 
6. The FINISH will end the task in case the user says that there is no feedback

Given the following user request, respond with the worker to act next.
If the worker is ToolCaller you must also add what it needs to do:
Example: ToolCall, to retrieve data from athena. 

For the other workers, only the name is to be generated.
Each worker will perform a task and respond with their results and status.
"""
architect_prompt = ChatPromptTemplate.from_messages([
    ("system", architect_instructions), 
    MessagesPlaceholder(variable_name="messages")])
architect_chain = architect_prompt | LLM

# Report Maker
report_instructions = """
You are responsible for generating a report on the data you receive. 
If there is feedback you must take it into account.
The data: {data}
The context: {context}
Output the following: This is a dummy report.
If there is feedback: {feedback}
Then add the following to the above: There is feedback
"""
report_prompt = ChatPromptTemplate.from_messages(
    [("system", report_instructions), MessagesPlaceholder(variable_name="messages")])
report_chain = report_prompt | LLM

tool_caller_instructions = """
You are an assitant that works together with other agents to produce a report. Your task is to use the tools
you have access to provide the data to the other agents.

You must do everything you are tasked with before ending.
"""
tool_caller_prompt = ChatPromptTemplate.from_messages(
    [("system", tool_caller_instructions), MessagesPlaceholder(variable_name="messages")])
tool_caller_chain = tool_caller_prompt | LLM.bind_tools([sales_records, context_retriever])

# Nodes
def assistant(state: State) -> State:
    print("*************In the assitant node*************")
    result = architect_chain.invoke({"messages": state["messages"]})
    state["messages"].append(result)
    return state

def tool_caller(state: State) -> State:
    print("*************In the Tool Caller node*************")
    last_message = state["messages"][-1]
    output = tool_caller_chain.invoke({"messages": [
        HumanMessage(f"Given the following messages {last_message}, use the appropriate tool you have access to.")]})
    state["messages"].append(output)
    return state

def reporter(state: State) -> State:
    print("*************In the report node*************")
    feedback = feedback = state.get("feedback", "No feedback")
    report = report_chain.invoke({"messages": [HumanMessage("Generate a report given the data, context and feedback provided")],
                                  "context": state["context"],
                                  "data": state["client_data"],
                                  "feedback": feedback})
    state["messages"].append(report)
    return state

def human_feedback_node(state: State) -> State:
    print("*************In the human feedback Node*************")
    report_message = state["messages"][-1]    
    while True:
        choice = input("Would you like to add feedback? (Yes or No)")
        if choice in ["Yes", "No"]:
            break
        if choice == "No":
            state["messages"].append(HumanMessage(content="No feedback to add. Finish."))
        print("Invalid choice, please try again (Yes or No)")

    if choice == "Yes":
        state["feedback"] = True
        while True:
                feedback = input("Please specify what feedback you would like to add: ").strip()
                if feedback:
                    state["messages"].append(HumanMessage(content=feedback))
                    state["feedback"] = feedback
                    break
                print("Request cannot be empty. Please try again.")
    else:
        state["feedback"] = False
    return state

def general_router(state: State):
    print("*************In the General router Node*************")
    last_message = state["messages"][-1]
    if "ToolCaller" in last_message.content:
        return "ToolCaller"
    elif last_message.content == "ReportMaker":
        return "ReportMaker"
    elif last_message.content == "FeedbackRequester":
        return "FeedbackRequester"
    elif last_message.content in ["There is no feedback.", "FINISH"]:
        return "__end__"

def tool_router(state: State) -> State:
    print("*************In the Tool router Node*************")
    last_message = state["messages"][-1]
    if last_message.tool_calls:
        return "Tools"
    else:
        return "Architect"

def run_tool(state: State, tools: dict) -> State:
    print("*************running the tool*************")
    tool_calls = state["messages"][-1].tool_calls
    for tool_call in tool_calls:
        tool = tools[tool_call["name"]]
        result = tool.invoke(tool_call["args"])
        state["messages"].append(ToolMessage(content=str(result), 
                                             tool_call_id= tool_call["id"]))
        if tool.name == "sales_records":
            state["client_data"] = str(result)
        if tool.name == "context_retriever":
            state["context"] = result
    return state

workflow = StateGraph(State)
workflow.add_node("Architect", assistant)
workflow.add_node("ToolCaller", tool_caller)
workflow.add_node("ReportMaker", reporter)
workflow.add_node("FeedbackRequester", human_feedback_node)
workflow.add_node("Tools", lambda state: run_tool(state, tools={"sales_records":sales_records, "context_retriever": context_retriever}))

workflow.add_edge(START, "Architect")
workflow.add_conditional_edges(
    "Architect",
    general_router,
    {
        "__end__": END,
        "ToolCaller": "ToolCaller",
        "ReportMaker": "ReportMaker",
        "FeedbackRequester": "FeedbackRequester"
    }
)
workflow.add_conditional_edges(
    "ToolCaller",
    tool_router,
    {
        "Architect": "Architect",
        "Tools": "Tools",
    }
)
workflow.add_edge("Tools", "Architect")
workflow.add_edge("ReportMaker", "Architect")
workflow.add_edge("FeedbackRequester", "Architect")

graph = workflow.compile()

events = graph.invoke({"messages": HumanMessage(
    content="Give me a report for the following, start date:1998-01-10 00:00:00.000 and end_date:2005-09-13 00:00:00.000")})

Output:

*************In the assitant node*************
*************In the General router Node*************
*************In the Tool Caller node*************
*************In the Tool router Node*************
*************running the tool*************
*************In the assitant node*************
---------------------------------------------------------------------------
ValidationException                       Traceback (most recent call last)
Cell In[1], line 264
    260 workflow.add_edge("FeedbackRequester", "Architect")
    262 graph = workflow.compile()
--> 264 events = graph.invoke({"messages": HumanMessage(
    265     content="Give me a report for the following, start date:1998-01-10 00:00:00.000 and end_date:2005-09-13 00:00:00.000")})

File [~\PycharmProjects\chatbot-reporting\chatbotenv\lib\site-packages\langgraph\pregel\__init__.py:1551](http://localhost:8888/lab/tree/~/PycharmProjects/chatbot-reporting/chatbotenv/lib/site-packages/langgraph/pregel/__init__.py#line=1550), in Pregel.invoke(self, input, config, stream_mode, output_keys, interrupt_before, interrupt_after, debug, **kwargs)
   1549 else:
   1550     chunks = []
-> 1551 for chunk in self.stream(
   1552     input,
   1553     config,
   1554     stream_mode=stream_mode,
   1555     output_keys=output_keys,
   1556     interrupt_before=interrupt_before,
   1557     interrupt_after=interrupt_after,
   1558     debug=debug,
   1559     **kwargs,
   1560 ):
   1561     if stream_mode == "values":
   1562         latest = chunk

File [~\PycharmProjects\chatbot-reporting\chatbotenv\lib\site-packages\langgraph\pregel\__init__.py:1290](http://localhost:8888/lab/tree/~/PycharmProjects/chatbot-reporting/chatbotenv/lib/site-packages/langgraph/pregel/__init__.py#line=1289), in Pregel.stream(self, input, config, stream_mode, output_keys, interrupt_before, interrupt_after, debug, subgraphs)
   1279     # Similarly to Bulk Synchronous Parallel / Pregel model
   1280     # computation proceeds in steps, while there are channel updates
   1281     # channel updates from step N are only visible in step N+1
   1282     # channels are guaranteed to be immutable for the duration of the step,
   1283     # with channel updates applied only at the transition between steps
   1284     while loop.tick(
   1285         input_keys=self.input_channels,
   1286         interrupt_before=interrupt_before_,
   1287         interrupt_after=interrupt_after_,
   1288         manager=run_manager,
   1289     ):
-> 1290         for _ in runner.tick(
   1291             loop.tasks.values(),
   1292             timeout=self.step_timeout,
   1293             retry_policy=self.retry_policy,
   1294             get_waiter=get_waiter,
   1295         ):
   1296             # emit output
   1297             yield from output()
   1298 # emit output

File [~\PycharmProjects\chatbot-reporting\chatbotenv\lib\site-packages\langgraph\pregel\runner.py:56](http://localhost:8888/lab/tree/~/PycharmProjects/chatbot-reporting/chatbotenv/lib/site-packages/langgraph/pregel/runner.py#line=55), in PregelRunner.tick(self, tasks, reraise, timeout, retry_policy, get_waiter)
     54 t = tasks[0]
     55 try:
---> 56     run_with_retry(t, retry_policy)
     57     self.commit(t, None)
     58 except Exception as exc:

File [~\PycharmProjects\chatbot-reporting\chatbotenv\lib\site-packages\langgraph\pregel\retry.py:29](http://localhost:8888/lab/tree/~/PycharmProjects/chatbot-reporting/chatbotenv/lib/site-packages/langgraph/pregel/retry.py#line=28), in run_with_retry(task, retry_policy)
     27 task.writes.clear()
     28 # run the task
---> 29 task.proc.invoke(task.input, config)
     30 # if successful, end
     31 break

File [~\PycharmProjects\chatbot-reporting\chatbotenv\lib\site-packages\langgraph\utils\runnable.py:385](http://localhost:8888/lab/tree/~/PycharmProjects/chatbot-reporting/chatbotenv/lib/site-packages/langgraph/utils/runnable.py#line=384), in RunnableSeq.invoke(self, input, config, **kwargs)
    383 context.run(_set_config_context, config)
    384 if i == 0:
--> 385     input = context.run(step.invoke, input, config, **kwargs)
    386 else:
    387     input = context.run(step.invoke, input, config)

File [~\PycharmProjects\chatbot-reporting\chatbotenv\lib\site-packages\langgraph\utils\runnable.py:167](http://localhost:8888/lab/tree/~/PycharmProjects/chatbot-reporting/chatbotenv/lib/site-packages/langgraph/utils/runnable.py#line=166), in RunnableCallable.invoke(self, input, config, **kwargs)
    165 else:
    166     context.run(_set_config_context, config)
--> 167     ret = context.run(self.func, input, **kwargs)
    168 if isinstance(ret, Runnable) and self.recurse:
    169     return ret.invoke(input, config)

Cell In[1], line 149, in assistant(state)
    147 def assistant(state: State) -> State:
    148     print("*************In the assitant node*************")
--> 149     result = architect_chain.invoke({"messages": state["messages"]})
    150     state["messages"].append(result)
    151     return state

File [~\PycharmProjects\chatbot-reporting\chatbotenv\lib\site-packages\langchain_core\runnables\base.py:3024](http://localhost:8888/lab/tree/~/PycharmProjects/chatbot-reporting/chatbotenv/lib/site-packages/langchain_core/runnables/base.py#line=3023), in RunnableSequence.invoke(self, input, config, **kwargs)
   3022             input = context.run(step.invoke, input, config, **kwargs)
   3023         else:
-> 3024             input = context.run(step.invoke, input, config)
   3025 # finish the root run
   3026 except BaseException as e:

File [~\PycharmProjects\chatbot-reporting\chatbotenv\lib\site-packages\langchain_core\language_models\chat_models.py:286](http://localhost:8888/lab/tree/~/PycharmProjects/chatbot-reporting/chatbotenv/lib/site-packages/langchain_core/language_models/chat_models.py#line=285), in BaseChatModel.invoke(self, input, config, stop, **kwargs)
    275 def invoke(
    276     self,
    277     input: LanguageModelInput,
   (...)
    281     **kwargs: Any,
    282 ) -> BaseMessage:
    283     config = ensure_config(config)
    284     return cast(
    285         ChatGeneration,
--> 286         self.generate_prompt(
    287             [self._convert_input(input)],
    288             stop=stop,
    289             callbacks=config.get("callbacks"),
    290             tags=config.get("tags"),
    291             metadata=config.get("metadata"),
    292             run_name=config.get("run_name"),
    293             run_id=config.pop("run_id", None),
    294             **kwargs,
    295         ).generations[0][0],
    296     ).message

File [~\PycharmProjects\chatbot-reporting\chatbotenv\lib\site-packages\langchain_core\language_models\chat_models.py:786](http://localhost:8888/lab/tree/~/PycharmProjects/chatbot-reporting/chatbotenv/lib/site-packages/langchain_core/language_models/chat_models.py#line=785), in BaseChatModel.generate_prompt(self, prompts, stop, callbacks, **kwargs)
    778 def generate_prompt(
    779     self,
    780     prompts: list[PromptValue],
   (...)
    783     **kwargs: Any,
    784 ) -> LLMResult:
    785     prompt_messages = [p.to_messages() for p in prompts]
--> 786     return self.generate(prompt_messages, stop=stop, callbacks=callbacks, **kwargs)

File [~\PycharmProjects\chatbot-reporting\chatbotenv\lib\site-packages\langchain_core\language_models\chat_models.py:643](http://localhost:8888/lab/tree/~/PycharmProjects/chatbot-reporting/chatbotenv/lib/site-packages/langchain_core/language_models/chat_models.py#line=642), in BaseChatModel.generate(self, messages, stop, callbacks, tags, metadata, run_name, run_id, **kwargs)
    641         if run_managers:
    642             run_managers[i].on_llm_error(e, response=LLMResult(generations=[]))
--> 643         raise e
    644 flattened_outputs = [
    645     LLMResult(generations=[res.generations], llm_output=res.llm_output)  # type: ignore[list-item]
    646     for res in results
    647 ]
    648 llm_output = self._combine_llm_outputs([res.llm_output for res in results])

File [~\PycharmProjects\chatbot-reporting\chatbotenv\lib\site-packages\langchain_core\language_models\chat_models.py:633](http://localhost:8888/lab/tree/~/PycharmProjects/chatbot-reporting/chatbotenv/lib/site-packages/langchain_core/language_models/chat_models.py#line=632), in BaseChatModel.generate(self, messages, stop, callbacks, tags, metadata, run_name, run_id, **kwargs)
    630 for i, m in enumerate(messages):
    631     try:
    632         results.append(
--> 633             self._generate_with_cache(
    634                 m,
    635                 stop=stop,
    636                 run_manager=run_managers[i] if run_managers else None,
    637                 **kwargs,
    638             )
    639         )
    640     except BaseException as e:
    641         if run_managers:

File [~\PycharmProjects\chatbot-reporting\chatbotenv\lib\site-packages\langchain_core\language_models\chat_models.py:851](http://localhost:8888/lab/tree/~/PycharmProjects/chatbot-reporting/chatbotenv/lib/site-packages/langchain_core/language_models/chat_models.py#line=850), in BaseChatModel._generate_with_cache(self, messages, stop, run_manager, **kwargs)
    849 else:
    850     if inspect.signature(self._generate).parameters.get("run_manager"):
--> 851         result = self._generate(
    852             messages, stop=stop, run_manager=run_manager, **kwargs
    853         )
    854     else:
    855         result = self._generate(messages, stop=stop, **kwargs)

File [~\PycharmProjects\chatbot-reporting\chatbotenv\lib\site-packages\langchain_aws\chat_models\bedrock_converse.py:495](http://localhost:8888/lab/tree/~/PycharmProjects/chatbot-reporting/chatbotenv/lib/site-packages/langchain_aws/chat_models/bedrock_converse.py#line=494), in ChatBedrockConverse._generate(self, messages, stop, run_manager, **kwargs)
    491 bedrock_messages, system = _messages_to_bedrock(messages)
    492 params = self._converse_params(
    493     stop=stop, **_snake_to_camel_keys(kwargs, excluded_keys={"inputSchema"})
    494 )
--> 495 response = self.client.converse(
    496     messages=bedrock_messages, system=system, **params
    497 )
    498 response_message = _parse_response(response)
    499 return ChatResult(generations=[ChatGeneration(message=response_message)])

File [~\PycharmProjects\chatbot-reporting\chatbotenv\lib\site-packages\botocore\client.py:565](http://localhost:8888/lab/tree/~/PycharmProjects/chatbot-reporting/chatbotenv/lib/site-packages/botocore/client.py#line=564), in ClientCreator._create_api_method.<locals>._api_call(self, *args, **kwargs)
    561     raise TypeError(
    562         f"{py_operation_name}() only accepts keyword arguments."
    563     )
    564 # The "self" in this scope is referring to the BaseClient.
--> 565 return self._make_api_call(operation_name, kwargs)

File [~\PycharmProjects\chatbot-reporting\chatbotenv\lib\site-packages\botocore\client.py:1017](http://localhost:8888/lab/tree/~/PycharmProjects/chatbot-reporting/chatbotenv/lib/site-packages/botocore/client.py#line=1016), in BaseClient._make_api_call(self, operation_name, api_params)
   1013     error_code = error_info.get("QueryErrorCode") or error_info.get(
   1014         "Code"
   1015     )
   1016     error_class = self.exceptions.from_code(error_code)
-> 1017     raise error_class(parsed_response, operation_name)
   1018 else:
   1019     return parsed_response

ValidationException: An error occurred (ValidationException) when calling the Converse operation: The toolConfig field must be defined when using toolUse and toolResult content blocks.

Therefore, this issue was opened since there does not seem to be a reasonable way (not present in the documentation) to pass information through the graph.

[3coins]

@DiogoPM9 The problem with the assistant not working with the tool response might be because it has no knowledge of the tools (no bind_tools on assistant LLM).

Also, I feel like the graph here can be simplified by using the prebuilt react agent. For example, see here. https://langchain-ai.github.io/langgraph/how-tos/tool-calling/#react-agent

Here is another example that showcases how to use the interrupt feature in graph to get human input. https://langchain-ai.github.io/langgraph/how-tos/human_in_the_loop/wait-user-input/

[DiogoPM9]

@3coins Thank you for the reply!

My issue with your first point is that I am binding tools to an LLM that will never use them. As per this Langgraph guide: Multi-agent supervisor, the supervisor node does not have access to any tools. Additionally the code works with an OpenAI LLM, hence my concern that there is something specific about ChatBedrockCovnerse . Perhaps this is just how Anthropic or Mistral models are configured?

Regarding your other two points, absolutely agree, I have since implemented some changes that simplify the code, however the pre-built react agent is quite restrictive with custom workflows so I avoided it.

EDIT:

I just tried binding the tools to the architect, as expected, it becomes confused and instead of delegating the task to other nodes it makes tool calls (which is not meant to do).

[tysoekong]

We've found the same issue.

Bedrock (Converse in my case) requires that the tool definitions to complete a chat, are sent in every inference request.

It is not satisfied that you reference a tool result, in a chat that:

• the tool has already been answered
• but is not defined in toolConfig

This seems like a Bedrock bug honestly.

[3coins]

@DiogoPM9 @tysoekong Thanks for those inputs. I am going to work on a simplified sample to investigate this further. This could be a Bedrock service issue, but I think if we can come up with a simple reproducible example, it will be easier to find a workaround (short-term) or work with the Bedrock team to figure out a path forward (long term). I have added this issue to the next milestone, due next Thursday, hopefully will have something concrete in a few days.