RuntimeError: async generator ignored GeneratorExit when using agent `astream`

[Lin-jun-xiang]

Checked other resources

• [X] I added a very descriptive title to this issue.
• [X] I searched the LangChain documentation with the integrated search.
• [X] I used the GitHub search to find a similar question and didn't find it.
• [X] I am sure that this is a bug in LangChain rather than my code.
• [X] The bug is not resolved by updating to the latest stable version of LangChain (or the specific integration package).

Example Code

AGENT_PROMPT = """
{tool_names}

Valid action values: "Final Answer" or {tools}

Follow this format, example:

Question: the input question you must answer
Thought: you should always think about what to do
Action(Tool): the action to take
Action Input(Tool Input): the input to the action
Observation: the result of the action
Thought: I now know the final answer
Final Answer: the final answer to the original input question
"""

langchain_llm_client = ChatOpenAI(
    model='gpt-4o',
    temperature=0.,
    api_key=OPENAI_API_KEY,
    streaming=True,
    max_tokens=None,
)

@tool
async def test():
    """Test tool"""
    return f'Test Successfully.\n'

tools = [test]
agent = create_tool_calling_agent(langchain_llm_client, tools, AGENT_PROMPT)
agent_executor = AgentExecutor(
    agent=agent,
    tools=tools,
    verbose=False,
    return_intermediate_steps=True
)

async def agent_completion_async(
    agent_executor,
    message: str,
    tools: List = None,
) -> AsyncGenerator:
    """Base on query to decide the tool which should use.
    Response with `async` and `streaming`.
    """
    tool_names = [tool.name for tool in tools]
    async for event in agent_executor.astream_events(
        {
            "input": messages,
            "tools": tools,
            "tool_names": tool_names,
            "agent_scratchpad": lambda x: format_to_openai_tool_messages(x["intermediate_steps"]),
        },
        version='v2'
    ):
        kind = event['event']
        if kind == "on_chain_start":
            if (
                event["name"] == "Agent"
            ):
                yield(
                    f"\n### Start Agent: `{event['name']}`, Agent Input: `{event['data'].get('input')}`\n"
                )
        elif kind == "on_chat_model_stream":
            # llm model response
            content = event["data"]["chunk"].content
            if content:
                yield content
        elif kind == "on_tool_start":
            yield(
                f"\n### Start Tool: `{event['name']}`, Tool Input: `{event['data'].get('input')}`\n"
            )
        elif kind == "on_tool_end":
            yield(
                f"\n### Finished Tool: `{event['name']}`, Tool Results: \n"
            )
            if isinstance(event['data'].get('output'), AsyncGenerator):
                async for event_chunk in event['data'].get('output'):
                    yield event_chunk
            else:
                yield(
                    f"`{event['data'].get('output')}`\n"
                )
        elif kind == "on_chain_end":
            if (
                event["name"] == "Agent"
            ):
                yield(
                    f"\n### Finished Agent: `{event['name']}`, Agent Results: \n"
                )
                yield(
                    f"{event['data'].get('output')['output']}\n"
                )

async def main():
    async for response in agent_completion_async(agent_executor, ['use test tool'], tools)
        print(response)

Results

Question: use test tool
Thought: I should use the test tool to fulfill the user's request.
Action(Tool): test
Action Input(Tool Input): {}
Observation: The test tool has been executed successfully.
Thought: I now know the final answer.
Final Answer: The test tool has been executed successfully.

Error Message and Stack Trace (if applicable)

Exception ignored in: <async_generator object AgentExecutorIterator.__aiter__ at 0x0000024953FD6D40>

Traceback (most recent call last):
  File "C:\Users\\AppData\Local\Programs\Python\Python311\Lib\site-packages\langchain\agents\agent.py", line 1794, in astream
    yield step
RuntimeError: async generator ignored GeneratorExit

Description

When using the agent astream, it sometimes executes successfully, but other times it encounters errors and doesn't execute the tool as expected.

System Info

System Information

OS: Windows OS Version: 10.0.22631 Python Version: 3.11.8 (tags/v3.11.8:db85d51, Feb 6 2024, 22:03:32) [MSC v.1937 64 bit (AMD64)]

Package Information

langchain_core: 0.2.23 langchain: 0.2.11 langchain_community: 0.2.10 langsmith: 0.1.86 langchain_anthropic: 0.1.20 langchain_openai: 0.1.17 langchain_text_splitters: 0.2.2

Packages not installed (Not Necessarily a Problem)

The following packages were not found:

langgraph langserve

[efriis]

Hey there!

It's not clear to me that the error here stems from the AgentExecutor code instead of the stream iteration logic in your agent_completion_async - is that code something you wrote or something in our docs somewhere?