AgentOps-AI / agentops

Python SDK for AI agent monitoring, LLM cost tracking, benchmarking, and more. Integrates with most LLMs and agent frameworks like CrewAI, Langchain, and Autogen
https://agentops.ai
MIT License
1.98k stars 191 forks source link

Issue with “Agent stopped due to iteration limit or time limit” in CrewAI with AgentOps Integration #363

Open nickzren opened 2 months ago

nickzren commented 2 months ago

Description: I am encountering an issue with the CrewAI integration when using AgentOps. Specifically, when integrating AgentOps into my CrewAI project, the agents consistently stop with the message "Agent stopped due to iteration limit or time limit," even though the tasks should complete without hitting these limits.

This issue does not occur when I run the same code without AgentOps integration. The tasks execute successfully, and the agents complete their interactions as expected.

Steps to Reproduce:

  1. Set up a simple CrewAI script with AgentOps integration.
  2. Initialize a session using agentops.init.
  3. Define simple agents and tasks using CrewAI.
  4. Run the script.
  5. Observe the issue where agents stop due to an iteration or time limit.

Sample Code:

import os
import json
import agentops
from crewai import Agent, Task, Crew, Process
from langchain_openai import ChatOpenAI
from dotenv import load_dotenv

# Load environment variables
load_dotenv()
openai_api_key = os.getenv('OPENAI_API_KEY')
agentops_api_key = os.getenv('AGENTOPS_API_KEY')

if not openai_api_key:
    raise EnvironmentError("OPENAI_API_KEY not found in environment variables.")
if not agentops_api_key:
    raise EnvironmentError("AGENTOPS_API_KEY not found in environment variables.")

# Initialize AgentOps session
agentops.init(api_key=agentops_api_key, default_tags=["crewai-simple"])

# Define the Simple Generator Agent
simple_generator = Agent(
    role='Simple Generator',
    goal='Generate a simple message based on input.',
    allow_delegation=False,  # No delegation to ensure single interaction
    backstory='You are a simple agent for generating messages.',
    llm=ChatOpenAI(model_name='gpt-4o-mini', openai_api_key=openai_api_key),
    max_iter=1  # Limit to one iteration
)

# Define the Simple Checker Agent
simple_checker = Agent(
    role='Simple Checker',
    goal='Check the generated message for any issues.',
    allow_delegation=False,  # No delegation to ensure single interaction
    backstory='You are a simple agent for checking messages.',
    llm=ChatOpenAI(model_name='gpt-4o-mini', openai_api_key=openai_api_key),
    max_iter=1  # Limit to one iteration
)

# Define the Simple Tasks
generate_message_task = Task(
    description="Generate a simple message based on input.",
    expected_output="A simple message generated by the agent.",
    agent=simple_generator
)

check_message_task = Task(
    description="Check the generated message for issues.",
    expected_output="If no issues, return 'Pass'. Otherwise, describe the issue.",
    agent=simple_checker,
    context=[generate_message_task]
)

# Function to save the final message, recorded with AgentOps
@agentops.record_action('save_final_message function execution')
def save_final_message(task_output):
    output_path = 'data/output/message.txt'
    output_data = json.loads(task_output.raw)
    if 'final_message' in output_data:
        with open(output_path, 'w') as file:
            file.write(output_data['final_message'])
        print(f"Final message has been written to {output_path}.")

# Create and execute the crew
crew = Crew(
    agents=[simple_generator, simple_checker],
    tasks=[generate_message_task, check_message_task],
    manager_llm=simple_generator.llm,
    process=Process.sequential,  # Ensure tasks are executed in sequence
    verbose=True
)

crew.kickoff(
    inputs={'message': 'Hello, this is a test message!'}
)

# End the AgentOps session
agentops.end_session("Success")

Expected Behavior: Agents should complete their tasks without prematurely stopping due to iteration or time limits.

Actual Behavior: Agents stop with the message "Agent stopped due to iteration limit or time limit," even though tasks are not expected to reach these limits.

Additional Notes:

Please let me know if further information is needed to assist with debugging this issue.

Thank you!

areibman commented 2 months ago

Hey @nickzren -- We're taking a look. Can you verify:

nickzren commented 2 months ago

Thanks

AgentOps: 0.3.9 CrewAI: 0.51.1

areibman commented 2 months ago

Thanks

AgentOps: 0.3.9 CrewAI: 0.51.1

Can you upgrade to 0.3.10? It's working for me in Jupyter notebook

image image
nickzren commented 2 months ago

This is odd, I have upgraded AgentOps to 0.3.10, but still getting the same error.

Screenshot 2024-08-20 at 10 23 51 AM
areibman commented 2 months ago

This is odd, I have upgraded AgentOps to 0.3.10, but still getting the same error.

Screenshot 2024-08-20 at 10 23 51 AM

Seems to be a LangChain error. No clear answer, but it may have to do something with your OpenAI rate limits.

Can you try swapping models?

areibman commented 1 month ago

Hey @nickzren -- any updates on this from you? Are you running into this issue with the latest version of Crew + AgentOps?