Structural problems with tool calls

[drdsgvo]

Newest crewai version: With state of the art Open-Source Models, that can run locally, crewai fails. Is there a way to intercept the tool call? There are so many errors when tools should be called. This is a showstopper and makes crewai unusable for serious tasks other than call LLMs in a sort of orchestrated way. It is paramount to be able to fix wrong calls and correct them automatically. Will this be planned or is it already possible?

[lorenzejay]

@drdsgvo Do you have code example?

[drdsgvo]

@drdsgvo Do you have code example?

E.g. with this tool definition (or original DuckDuckGoSearchRun) @tool('DuckDuckGoSearch') def search_tool(search_query: str): #also tried with param name "query" """Search the web for information on a given
return DuckDuckGoSearchRun().run(search_query,region="de-de")

or with @tool('DirectoryReadTool') def drdsgvo_docs_tool(**kwargs: Any): """ Search the known docments for information on a given topic """

...

calling using DirectoryReadTool

then using these tools for two agents, one tool per agent these tools come into play for agents that act as researchers, after an agent acting as a planner.

I do not think that it is about code examples here. My question is whether one can handle badly formulated agent ouput that serves as basis for tooling? If that isn not possible, then crewai is not really useful with current local models.

model: llama3 (8b)

[AIMPED]

Hey @drdsgvo, here an example which fails using other models than GPT-4o. I used llama3 hosted by groq. CrewAI tries to delegate to a fictional co- worker for the story creation. For translation it tries to use a translation tool.

This is just an example, I have other use- cases where the tool gets called correctly, the "Thought" is correct but crewAI enters in a loop which fails at max. iterations.

# crewai imports
from crewai import Agent, Task, Crew, Process
from crewai_tools import tool
from crewai.tasks.task_output import TaskOutput

# langchain imports
from langchain_groq import ChatGroq
from langchain_core.callbacks import CallbackManager, StreamingStdOutCallbackHandler
from langchain_openai import ChatOpenAI

# other imports
import dotenv
import os

# load API_Keys
_ = dotenv.load_dotenv()

# callback manager for streaming output
callback_manager = CallbackManager([StreamingStdOutCallbackHandler()])

llm = ChatGroq(
    api_key=os.environ.get("GROQ_API_KEY"),
    model="llama3-70b-8192",
    callback_manager=callback_manager,
    temperature=0.0
)
# llm = ChatOpenAI(
#     api_key=os.getenv("openAI_API_KEY"),
#     model="gpt-4o",
#     callback_manager=callback_manager,
#     temperature=0.0
# )

@tool("openFile")
def open_file(path_to_file: str):
    """
    This tool is useful if you have to check the contents of a given text file
    """
    with open(path_to_file, "rt") as f:
        return f.read()

def callback_function(output: TaskOutput):
    # Do something after the task is completed
    with open("initial_story.txt", "wt") as f:
        f.write(f"{output.raw_output}")

comedian = Agent(
    role="comedian",
    backstory="You are the host of a famous late night show and known for your funny stories.",
    goal='funny story',
    verbose=True,
    llm=llm,
    tools=[open_file],
)

translator = Agent(
    role="translator",
    backstory="You are a experienced translator for translating text from English to Spanish",
    goal='translated text',
    verbose=True,
    llm=llm,
)

create_story = Task(
    description="Tell a funny story about the following topic {topic}. Make it short."
                "You have a personal archive with all stories for all topics you ever told before, this is the path to it: {file_path_to_stories}"
                "if you find a story about the {topic}, re use it and don't tell a new one",
    callback=callback_function,
    expected_output="a funny story",
    agent=comedian
)

translate_story = Task(
    description="translate the given context to spanish",
    expected_output="a good translation of the given context",
    agent=translator,
    context=[create_story],
    output_file="spanish_story.txt"
)

crew = Crew(
  agents=[comedian, translator],
  tasks=[create_story, translate_story],
  process=Process.sequential,
  verbose=2,
)

if __name__ == "__main__":
    crew.kickoff(inputs={
        "topic": "cat",
        "file_path_to_stories": "./archive.txt"}
    )

archive.txt:

Alright, folks, let me tell you about the time I went snorkeling and had a run-in with a jellyfish. So there I was, floating in the crystal-clear waters of the Caribbean, feeling like a majestic sea creature myself. Suddenly, I felt this weird, tingling sensation on my arm. I looked down and saw a jellyfish just chilling there, like it was on vacation too.

Now, I don't know if you've ever seen a jellyfish up close, but they look like a cross between a plastic bag and a lava lamp. This one was just bobbing around, minding its own business, but I was convinced it was out to get me. I started flailing around, trying to get away from it, but the more I moved, the more it seemed to follow me. It was like a bad date that just wouldn't take the hint.

Eventually, I managed to swim back to the boat, where the guide told me that the jellyfish was harmless and probably more scared of me than I was of it. But let me tell you, nothing makes you feel more ridiculous than realizing you've been outsmarted by a creature with no brain. So, the moral of the story is: if you ever find yourself face-to-face with a jellyfish, just remember, it's not out to get you—it's just trying to figure out what the heck you are too!

[github-actions[bot]]

This issue is stale because it has been open for 30 days with no activity. Remove stale label or comment or this will be closed in 5 days.

[github-actions[bot]]

This issue was closed because it has been stalled for 5 days with no activity.