"Action don't exist" when using Ollama.

[hiddenkirby]

I am running the example script from the readme. Other than my API keys, the only change I made was to have the script run off my local LLM.

When I run it, I get a Action 'Search the internet using the query "Latest Advancements in AI 2024".' don't exist, these are the only available Actions: Search the internet: Search the internet(search_query: 'string') - A tool that can be used to semantic search a query from a txt's content..

Any assistance would be appreciated.

Code below.

import os
from langchain_community.llms import Ollama
from crewai import Agent, Task, Crew, Process
from crewai_tools import SerperDevTool

os.environ["OPENAI_API_KEY"] = "<my_openai_api_key>"
os.environ["SERPER_API_KEY"] = "<my_serper_api_key>" # serper.dev API key

# You can choose to use a local model through Ollama for example. See https://docs.crewai.com/how-to/LLM-Connections/ for more information.

os.environ["OPENAI_API_BASE"] = 'http://localhost:11434/v1'
os.environ["OPENAI_MODEL_NAME"] ='llama2'  # Adjust based on available model
os.environ["OPENAI_API_KEY"] =''
llama2 = Ollama(model="llama2")

search_tool = SerperDevTool()

# Define your agents with roles and goals
researcher = Agent(
  role='Senior Research Analyst',
  goal='Uncover cutting-edge developments in AI and data science',
  backstory="""You work at a leading tech think tank.
  Your expertise lies in identifying emerging trends.
  You have a knack for dissecting complex data and presenting actionable insights.""",
  verbose=True,
  allow_delegation=False,
  tools=[search_tool],
  # You can pass an optional llm attribute specifying what mode you wanna use.
  # It can be a local model through Ollama / LM Studio or a remote
  # model like OpenAI, Mistral, Antrophic or others (https://docs.crewai.com/how-to/LLM-Connections/)
  #
  # import os
  # os.environ['OPENAI_MODEL_NAME'] = 'gpt-3.5-turbo'
  #
  # OR
  #
  # from langchain_openai import ChatOpenAI
  # llm=ChatOpenAI(model_name="gpt-3.5", temperature=0.7)
  llm=llama2
)
writer = Agent(
  role='Tech Content Strategist',
  goal='Craft compelling content on tech advancements',
  backstory="""You are a renowned Content Strategist, known for your insightful and engaging articles.
  You transform complex concepts into compelling narratives.""",
  verbose=True,
  allow_delegation=True,
  llm=llama2
)

# Create tasks for your agents
task1 = Task(
  description="""Conduct a comprehensive analysis of the latest advancements in AI in 2024.
  Identify key trends, breakthrough technologies, and potential industry impacts.""",
  expected_output="Full analysis report in bullet points",
  agent=researcher
)

task2 = Task(
  description="""Using the insights provided, develop an engaging blog
  post that highlights the most significant AI advancements.
  Your post should be informative yet accessible, catering to a tech-savvy audience.
  Make it sound cool, avoid complex words so it doesn't sound like AI.""",
  expected_output="Full blog post of at least 4 paragraphs",
  agent=writer
)

# Instantiate your crew with a sequential process
crew = Crew(
  agents=[researcher, writer],
  tasks=[task1, task2],
  verbose=2, # You can set it to 1 or 2 to different logging levels
)

# Get your crew to work!
result = crew.kickoff()

print("######################")
print(result)

Terminal output (I ctrl + c after the action message)

(.venv) me@my-mbp-2 crewai-testing % python crewai_test.py 
 [DEBUG]: == Working Agent: Senior Research Analyst
 [INFO]: == Starting Task: Conduct a comprehensive analysis of the latest advancements in AI in 2024.
  Identify key trends, breakthrough technologies, and potential industry impacts.

> Entering new CrewAgentExecutor chain...
Thought: I understand the task at hand, which is to conduct a comprehensive analysis of the latest advancements in AI in 2024. I will use the internet search tool to gather information and present my findings in a bullet point format.

Action: Search the internet using the query "Latest Advancements in AI 2024".
Action Input: {
"search_query": "Latest Advancements in AI 2024",
"tool": "Search the internet"
} 

Action 'Search the internet using the query "Latest Advancements in AI 2024".' don't exist, these are the only available Actions: Search the internet: Search the internet(search_query: 'string') - A tool that can be used to semantic search a query from a txt's content.

[hiddenkirby]

Using a different model (mistral) and simple tool example... (notice the Action and Action Input usage being incorrect)

import os
from crewai import Crew, Agent, Task
from tools.calculator_tools import CalculatorTools
from textwrap import dedent
from langchain_community.llms import Ollama

from dotenv import load_dotenv
load_dotenv()

# to use a local LLM
os.environ["OPENAI_API_BASE"] = 'http://localhost:11434/v1'
os.environ["OPENAI_MODEL_NAME"] ='mistral'  # Adjust based on available model
os.environ["OPENAI_API_KEY"] =''
mistral = Ollama(model="mistral")

class TestCrew:
    def __init__(self, firstNumber, secondNumber):
        self.firstNumber = firstNumber
        self.secondNumber = secondNumber

    def run(self):
        agent_with_calculate = Agent(
            role='Math Assistant',
            goal="""To assist with any and all mathematical calculations""",
            backstory="""
            You are a math teacher at a local high school.
            """,
            verbose=True,
            tools=[
                CalculatorTools.calculate
            ],
            llm=mistral
        )

        add_two_numbers = Task(
        description=dedent(f"""
            Given two numbers perform a multiplication calculation on them.

            First Number: {self.firstNumber}
            Second Number: {self.secondNumber}
        """),
        expected_output=dedent("""
            The result of the calculation.
        """),
        agent=agent_with_calculate
        )

        crew = Crew(
            agents=[
                agent_with_calculate
            ],
            tasks=[
                add_two_numbers
            ],
            verbose=True
        )

        result = crew.kickoff()
        return result

if __name__ == "__main__":
    print("--   Crew Start  --")
    print("----------------------------")
    firstNumber = input(
         dedent("""
             What is the first number?
                """))
    secondNumber = input(
         dedent("""
             What number would you like to multiply it by?
                """))
    analysis_crew = TestCrew(firstNumber, secondNumber)
    result = analysis_crew.run()
    print("------------- Result below ---------------")
    print(result)
    print("------------------------------------------")

And a tool definition example of

from langchain_community.tools import tool

class CalculatorTools():

  @tool("Make a calculation")
  def calculate(operation):
    """Useful to perform any mathematical calculations, 
    like sum, minus, multiplication, division, etc.
    The input to this tool should be a mathematical 
    expression, a couple examples are `200*7` or `5000/2*10`
    """
    print(f"\n\nUsing Tool to calculate: {operation}\n\n")
    return eval(operation)

Output below - notice the way it's mapping the tool's definition incorrectly. (Added print statements.)

 (.venv) me@my-mbp-2 crewai-testing % python [local_tool_test.py](http://local_tool_test.py/)
--   Crew Start  --
----------------------------

What is the first number?
3

What number would you like to multiply it by?
4
 [DEBUG]: == Working Agent: Math Assistant
 [INFO]: == Starting Task: 
Given two numbers perform a multiplication calculation on them.

First Number: 3
Second Number: 4

> Entering new CrewAgentExecutor chain...
 I need to perform a multiplication calculation using the given numbers: 3 and 4.

Action: Make a calculation
Action Input: {"operation": "multiplication", "num1": 3, "num2": 4}

Using Tool to calculate: multiplication

Using Tool to calculate: multiplication

Using Tool to calculate: multiplication

I encountered an error while trying to use the tool. This was the error: CalculatorTools.calculate() got an unexpected keyword argument 'num1'.
 Tool Make a calculation accepts these inputs: Make a calculation(operation) - Useful to perform any mathematical calculations, 
    like sum, minus, multiplication, division, etc.
    The input to this tool should be a mathematical 
    expression, a couple examples are `200*7` or `5000/2*10`

 Thought: I need to perform a multiplication calculation using the given numbers: 3 and 4.
Action: Make a calculation
Action Input: {"operation": "*", "num1": 3, "num2": 4}

Using Tool to calculate: *

Using Tool to calculate: *

Using Tool to calculate: *

I encountered an error while trying to use the tool. This was the error: CalculatorTools.calculate() got an unexpected keyword argument 'num1'.
 Tool Make a calculation accepts these inputs: Make a calculation(operation) - Useful to perform any mathematical calculations, 
    like sum, minus, multiplication, division, etc.
    The input to this tool should be a mathematical 
    expression, a couple examples are `200*7` or `5000/2*10`

To be clear, when commenting out the Ollama configuration, and removing the llm parameter from the Agent defn ... the tool is used properly. (Action and Action Input are then correct).

[hiddenkirby]

I'm still learning a lot, and I'm not a pro Python coder ... but I'll try to replicate this issue with just langchain_community.tools and langchain_community.llms and remove CrewAI from the equation.

[claudiocassimiro]

I’m trying to run my crewAi app with Mixtral:8x7b using ChatOpenAI and ollama

I’m create a docker compose with services of crewAi agents, db and ollama with an network bridge. All the things working fine, but ChatOpenAI don’t working with ollama base api on local :/

anybody have a suggestion to me? Thanks before

[kjenney]

This did work at one point, but now it's broken. This is very disappointing.

[nfoong]

I'm having the same issue as @hiddenkirby when trying to run the job-posting example from https://github.com/joaomdmoura/crewAI-examples. My only change is to agents.py, where I added the lines:

from langchain_community.llms import Ollama
ollama = Ollama(model='llama2')

and the line llm=ollama to each agent. E.g.:

def research_agent(self):
    return Agent(
        role='Research Analyst',
        goal='Analyze the company website and provided description to extract insights on culture, values, and specific needs.',
        tools=[web_search_tool, seper_dev_tool],
        backstory='Expert in analyzing company cultures and identifying key values and needs from various sources, including websites and brief descriptions.',
        verbose=True,
        llm=ollama
    )

It seems to load up the local llama2-7b model just fine via ollama, but it repeatedly throws the errors such as the one below.

> Entering new CrewAgentExecutor chain...
Thought: Based on the provided company website and description, I should analyze the content to understand the company's culture, values, and mission. I can leverage this information to attract the right candidates for the job role.

Action: Search in a specific website (search_query: 'crewAI', website: 'https://crewai.com') to gather insights on the company's culture and values.
Action Input: { "website": "https://crewai.com", "query": "crewAI" } 

Action 'Search in a specific website (search_query: 'crewAI', website: 'https://crewai.com') to gather insights on the company's culture and values.' don't exist, these are the only available Actions: Search in a specific website: Search in a specific website(search_query: 'string', website: 'string') - A tool that can be used to semantic search a query from a specific URL content.
Search the internet: Search the internet(search_query: 'string') - A tool that can be used to semantic search a query from a txt's content.
Delegate work to co-worker: Delegate work to co-worker(coworker: str, task: str, context: str) - Delegate a specific task to one of the following co-workers: ['Job Description Writer', 'Review and Editing Specialist']
The input to this tool should be the coworker, the task you want them to do, and ALL necessary context to exectue the task, they know nothing about the task, so share absolute everything you know, don't reference things but instead explain them.
Ask question to co-worker: Ask question to co-worker(coworker: str, question: str, context: str) - Ask a specific question to one of the following co-workers: ['Job Description Writer', 'Review and Editing Specialist']
The input to this tool should be the coworker, the question you have for them, and ALL necessary context to ask the question properly, they know nothing about the question, so share absolute everything you know, don't reference things but instead explain them.

After a few iterations, it seems to try to correct itself, but to no avail:

Thank you for correcting me! Here's my revised response:

Thought: I should analyze the company's website and description to understand their culture, values, and mission.
Action: Review the company's website and description to gather insights on their culture and values.
Action Input: N/A

Action 'Review the company's website and description to gather insights on their culture and values.' don't exist, these are the only available Act
ions: Search in a specific website: Search in a specific website(search_query: 'string', website: 'string') - A tool that can be used to semantic search a query from a specific URL content.
Search the internet: Search the internet(search_query: 'string') - A tool that can be used to semantic search a query from a txt's content.        
Delegate work to co-worker: Delegate work to co-worker(coworker: str, task: str, context: str) - Delegate a specific task to one of the following co-workers: ['Job Description Writer', 'Review and Editing Specialist']
The input to this tool should be the coworker, the task you want them to do, and ALL necessary context to exectue the task, they know nothing about the task, so share absolute everything you know, don't reference things but instead explain them.
Ask question to co-worker: Ask question to co-worker(coworker: str, question: str, context: str) - Ask a specific question to one of the following co-workers: ['Job Description Writer', 'Review and Editing Specialist']
The input to this tool should be the coworker, the question you have for them, and ALL necessary context to ask the question properly, they know nothing about the question, so share absolute everything you know, don't reference things but instead explain them.

Any help will be much appreciated, as I'd really like to get multi-agents working with local LLMs. Thanks!

[hiddenkirby]

I'm having the same issue as @hiddenkirby when trying to run the job-posting example from https://github.com/joaomdmoura/crewAI-examples. My only change is to agents.py, where I added the lines:

from langchain_community.llms import Ollama
ollama = Ollama(model='llama2')

and the line llm=ollama to each agent. E.g.:

def research_agent(self):
  return Agent(
      role='Research Analyst',
      goal='Analyze the company website and provided description to extract insights on culture, values, and specific needs.',
      tools=[web_search_tool, seper_dev_tool],
      backstory='Expert in analyzing company cultures and identifying key values and needs from various sources, including websites and brief descriptions.',
      verbose=True,
      llm=ollama
  )

It seems to load up the local llama2-7b model just fine via ollama, but it repeatedly throws the errors such as the one below.

> Entering new CrewAgentExecutor chain...
Thought: Based on the provided company website and description, I should analyze the content to understand the company's culture, values, and mission. I can leverage this information to attract the right candidates for the job role.

Action: Search in a specific website (search_query: 'crewAI', website: 'https://crewai.com') to gather insights on the company's culture and values.
Action Input: { "website": "https://crewai.com", "query": "crewAI" } 

Action 'Search in a specific website (search_query: 'crewAI', website: 'https://crewai.com') to gather insights on the company's culture and values.' don't exist, these are the only available Actions: Search in a specific website: Search in a specific website(search_query: 'string', website: 'string') - A tool that can be used to semantic search a query from a specific URL content.
Search the internet: Search the internet(search_query: 'string') - A tool that can be used to semantic search a query from a txt's content.
Delegate work to co-worker: Delegate work to co-worker(coworker: str, task: str, context: str) - Delegate a specific task to one of the following co-workers: ['Job Description Writer', 'Review and Editing Specialist']
The input to this tool should be the coworker, the task you want them to do, and ALL necessary context to exectue the task, they know nothing about the task, so share absolute everything you know, don't reference things but instead explain them.
Ask question to co-worker: Ask question to co-worker(coworker: str, question: str, context: str) - Ask a specific question to one of the following co-workers: ['Job Description Writer', 'Review and Editing Specialist']
The input to this tool should be the coworker, the question you have for them, and ALL necessary context to ask the question properly, they know nothing about the question, so share absolute everything you know, don't reference things but instead explain them.

After a few iterations, it seems to try to correct itself, but to no avail:

Thank you for correcting me! Here's my revised response:

Thought: I should analyze the company's website and description to understand their culture, values, and mission.
Action: Review the company's website and description to gather insights on their culture and values.
Action Input: N/A

Action 'Review the company's website and description to gather insights on their culture and values.' don't exist, these are the only available Act
ions: Search in a specific website: Search in a specific website(search_query: 'string', website: 'string') - A tool that can be used to semantic search a query from a specific URL content.
Search the internet: Search the internet(search_query: 'string') - A tool that can be used to semantic search a query from a txt's content.        
Delegate work to co-worker: Delegate work to co-worker(coworker: str, task: str, context: str) - Delegate a specific task to one of the following co-workers: ['Job Description Writer', 'Review and Editing Specialist']
The input to this tool should be the coworker, the task you want them to do, and ALL necessary context to exectue the task, they know nothing about the task, so share absolute everything you know, don't reference things but instead explain them.
Ask question to co-worker: Ask question to co-worker(coworker: str, question: str, context: str) - Ask a specific question to one of the following co-workers: ['Job Description Writer', 'Review and Editing Specialist']
The input to this tool should be the coworker, the question you have for them, and ALL necessary context to ask the question properly, they know nothing about the question, so share absolute everything you know, don't reference things but instead explain them.

Any help will be much appreciated, as I'd really like to get multi-agents working with local LLMs. Thanks!

@nfoong , try instantiating your LLM configuration like this instead.

from langchain_openai import ChatOpenAI
llm = ChatOpenAI(
    openai_api_base="http://localhost:11434/v1",
    openai_api_key="ollama",                 
    model_name="llama2"
)

[nfoong]

@hiddenkirby thanks for your reply. Were you able to get it working by doing this?

I just gave that a shot, but the issue still persists, the agent isn't able to choose the right Action as the semantics are wrong:

> Entering new CrewAgentExecutor chain...
Thought: Based on the information provided on the company website and hiring manager's domain, I should analyze the content to understand the company culture, values, and mission. I can use the available tools to gather insights and create a comprehensive report summarizing the findings.      

Action: Search in specific website (search_query: 'crewAI culture', website: 'https://crewai.com').
Action Input: None.

Action 'Search in specific website (search_query: 'crewAI culture', website: 'https://crewai.com').' don't exist, these are the only available Actions: Search in a specific website: Search in a specific website(search_query: 'string', website: 'string') - A tool that can be used to semantic search a query from a specific URL content[...]

It's probably meant to be something like this:

Action: Search in specific website(search_query: 'string', website: 'string').
Action Input: {search_query: 'crewAI culture', website: 'https://crewai.com'}

I don't have the expertise or time to debug this further. @kjenney mentioned that it was working before, so maybe a workaround would be to rollback the crewai version until I find a version where this works.

@joaomdmoura any ideas on what's going wrong will be appreciated! Thanks!

[PaulCutcliffe]

I'm getting the same issue if I run any of the demo scripts with a local LLM running on LM Server - it just displays this multiple times:

Thought: What are the latest advancements in AI? Action: [Search the internet]("Recent advancements in AI in 2024") Action Input: {}

Action '[Search the internet]("Recent advancements in AI in 2024")' don't exist, these are the only available Actions: Search the internet: Search the internet(search_query: 'string') - A tool that can be used to semantic search a query from a txt's content.

[hiddenkirby]

@nfoong

So, yes and no. Yes, I am able to successfully run scripts with tools against Ollama / Open-sourced LLMs. Here are the two main things I did. I do think there was a CrewAI/CrewAI[tools] update recently as well ... so the water is a bit muddy here (make sure you're running the latest versions of libraries).

1. Use the OpenAI wrapper instead of Ollama. (the idea is that OpenAI interface better supports Function Calling)

   from langchain_openai import ChatOpenAI
   llm = ChatOpenAI(
   openai_api_base="http://localhost:11434/v1",
   openai_api_key="ollama",                 
   model_name="mistral_tools"
   )

2. Try to tell the tool to focus on learning tools above all else. (No clue if this is effective or not.) As well as lower the temperature. a. created a modelfile

   FROM mistral:latest
   TEMPLATE """[INST] {{ .System }} {{ .Prompt }} [/INST]"""
   PARAMETER stop "[INST]"
   PARAMETER stop "[/INST]"
   PARAMETER temperature 0.2
   SYSTEM "You are an assistant proficient in understanding and learning how to call functions defined by the user."

   b. run ollama create mistral_tools-f ./Modelfile c. run ollama run mistral_tools

   Things are working for me. Occasionally, the LLM will call a function incorrectly as you have observed above, but it tries again and does a better job (eventually). I can move forward with this... somewhat.

[rodrigofarias-MECH]

@nfoong

So, yes and no. Yes, I am able to successfully run scripts with tools against Ollama / Open-sourced LLMs. Here are the two main things I did. I do think there was a CrewAI/CrewAI[tools] update recently as well ... so the water is a bit muddy here (make sure you're running the latest versions of libraries).

1. Use the OpenAI wrapper instead of Ollama. (the idea is that OpenAI interface better supports Function Calling)

from langchain_openai import ChatOpenAI
llm = ChatOpenAI(
    openai_api_base="http://localhost:11434/v1",
    openai_api_key="ollama",                 
    model_name="mistral_tools"
)

2. Try to tell the tool to focus on learning tools above all else. (No clue if this is effective or not.) As well as lower the temperature.
   a. created a modelfile

FROM mistral:latest
TEMPLATE """[INST] {{ .System }} {{ .Prompt }} [/INST]"""
PARAMETER stop "[INST]"
PARAMETER stop "[/INST]"
PARAMETER temperature 0.2
SYSTEM "You are an assistant proficient in understanding and learning how to call functions defined by the user."

b. run ollama create mistral_tools-f ./Modelfile c. run ollama run mistral_tools

Things are working for me. Occasionally, the LLM will call a function incorrectly as you have observed above, but it tries again and does a better job (eventually). I can move forward with this... somewhat.

Hello. Is there anyupdate on this matter? I tried your above solution but the problem persists.

my_llm= OpenAI(base_url="http://localhost:1234/v1", 
              api_key="ollama",
              model="mistral")

and the example of "action dont exist" (using ChatOpenAi gives the same result):

Action 'Delegate work to Technology Expert' don't exist, these are the only available Actions: Delegate 
work to co-worker: Delegate work to co-worker(coworker: str, task: str, context: str) - Delegate a specific task to one of the following co-workers: [Technology Expert, Business Development Consultant]       
The input to this tool should be the coworker, the task you want them to do, and ALL necessary context to exectue the task, they know nothing about the task, so share absolute everything you know, don't reference things but instead explain them.
Ask question to co-worker: Ask question to co-worker(coworker: str, question: str, context: str) - Ask a specific question to one of the following co-workers: [Technology Expert, Business Development Consultant]
The input to this tool should be the coworker, the question you have for them, and ALL necessary context to ask the question properly, they know nothing about the question, so share absolute everything you know, don't reference things but instead explain them.

The example above is the startup one from https://github.com/majacinka/crewai-experiments Im using LM Studio with Mistral Instruct. Tested other llm's, very similar results. Is there any tricks to properly run the LM Studio Server? Any link appreciated.

[hiddenkirby]

Honestly, I think there are a few pieces to the puzzle here: (I'm still learning)

1. The LLM of choice should be able to understand the concept of "function calling". Not all of them do. I can use mistral just fine but I think I need to turn down the temperature to 0.5 or less for it to get more consistent.
2. The API wrapper of choice should support "tools". I have seen the best results with ChatOpenAI from langchain_openai library. I have seen OllamaFunctions from langchain_experimental.llms as well but I just run with ChatOpenAI so i can swap between Gpt-4, gpt-3.5, and my local Ollama / Mistral.

I gained a good bit of clarity after going through this fella's tutorials on Langchain proper, but I'm still pretty new. ( https://www.youtube.com/playlist?list=PLcQVY5V2UY4Kat6vxC7ESzIIzHWdwlnak )

[kyuumeitai]

This did work at one point, but now it's broken. This is very disappointing.

I get it, maybe it's frustrating but this is open source, comments like this one must have that in mind, and maybe a PR that solves it... Or at least something to debug. That would be a constructive, significant contribution to a free, unpaid job that João makes here.

[kyuumeitai]

I have tested some of the suggestions, but the only thing that works for me is to use OpenHermes as local model. And yes, I've tried llama3, it doesn't do the work.

FROM openhermes
PARAMETER stop Result
PARAMETER temperature 0.6
PARAMETER num_ctx 8192

It's far from perfect but at least doesn't get stuck at the hidious Action don't exist message.

[rodrigofarias-MECH]

now Im focusing on task delegation only, without tools. But like @kyuumeitai said, something that is really producing less errors during execution is using 'temperature' values close to zero.

This "seems" to make the LLM follow more accurately the instructions inside crewai code. My only concern is that may limit the "creativity" of the model.

[hiddenkirby]

now Im focusing on task delegation only, without tools. But like @kyuumeitai said, something that is really producing less errors during execution is using 'temperature' values close to zero.

This "seems" to make the LLM follow more accurately the instructions inside crewai code. My only concern is that may limit the "creativity" of the model.

Right. That is basically a function of temperature. You might want to configure "creative" agents to use LLMs that don't have low temperature. Not all agents need to use the same LLM.

[github-actions[bot]]

This issue is stale because it has been open for 30 days with no activity. Remove stale label or comment or this will be closed in 5 days.

[github-actions[bot]]

This issue was closed because it has been stalled for 5 days with no activity.