Closed hiddenkirby closed 2 weeks ago
Using a different model (mistral) and simple tool example... (notice the Action and Action Input usage being incorrect)
import os
from crewai import Crew, Agent, Task
from tools.calculator_tools import CalculatorTools
from textwrap import dedent
from langchain_community.llms import Ollama
from dotenv import load_dotenv
load_dotenv()
# to use a local LLM
os.environ["OPENAI_API_BASE"] = 'http://localhost:11434/v1'
os.environ["OPENAI_MODEL_NAME"] ='mistral' # Adjust based on available model
os.environ["OPENAI_API_KEY"] =''
mistral = Ollama(model="mistral")
class TestCrew:
def __init__(self, firstNumber, secondNumber):
self.firstNumber = firstNumber
self.secondNumber = secondNumber
def run(self):
agent_with_calculate = Agent(
role='Math Assistant',
goal="""To assist with any and all mathematical calculations""",
backstory="""
You are a math teacher at a local high school.
""",
verbose=True,
tools=[
CalculatorTools.calculate
],
llm=mistral
)
add_two_numbers = Task(
description=dedent(f"""
Given two numbers perform a multiplication calculation on them.
First Number: {self.firstNumber}
Second Number: {self.secondNumber}
"""),
expected_output=dedent("""
The result of the calculation.
"""),
agent=agent_with_calculate
)
crew = Crew(
agents=[
agent_with_calculate
],
tasks=[
add_two_numbers
],
verbose=True
)
result = crew.kickoff()
return result
if __name__ == "__main__":
print("-- Crew Start --")
print("----------------------------")
firstNumber = input(
dedent("""
What is the first number?
"""))
secondNumber = input(
dedent("""
What number would you like to multiply it by?
"""))
analysis_crew = TestCrew(firstNumber, secondNumber)
result = analysis_crew.run()
print("------------- Result below ---------------")
print(result)
print("------------------------------------------")
And a tool definition example of
from langchain_community.tools import tool
class CalculatorTools():
@tool("Make a calculation")
def calculate(operation):
"""Useful to perform any mathematical calculations,
like sum, minus, multiplication, division, etc.
The input to this tool should be a mathematical
expression, a couple examples are `200*7` or `5000/2*10`
"""
print(f"\n\nUsing Tool to calculate: {operation}\n\n")
return eval(operation)
Output below - notice the way it's mapping the tool's definition incorrectly. (Added print statements.)
(.venv) me@my-mbp-2 crewai-testing % python [local_tool_test.py](http://local_tool_test.py/)
-- Crew Start --
----------------------------
What is the first number?
3
What number would you like to multiply it by?
4
[DEBUG]: == Working Agent: Math Assistant
[INFO]: == Starting Task:
Given two numbers perform a multiplication calculation on them.
First Number: 3
Second Number: 4
> Entering new CrewAgentExecutor chain...
I need to perform a multiplication calculation using the given numbers: 3 and 4.
Action: Make a calculation
Action Input: {"operation": "multiplication", "num1": 3, "num2": 4}
Using Tool to calculate: multiplication
Using Tool to calculate: multiplication
Using Tool to calculate: multiplication
I encountered an error while trying to use the tool. This was the error: CalculatorTools.calculate() got an unexpected keyword argument 'num1'.
Tool Make a calculation accepts these inputs: Make a calculation(operation) - Useful to perform any mathematical calculations,
like sum, minus, multiplication, division, etc.
The input to this tool should be a mathematical
expression, a couple examples are `200*7` or `5000/2*10`
Thought: I need to perform a multiplication calculation using the given numbers: 3 and 4.
Action: Make a calculation
Action Input: {"operation": "*", "num1": 3, "num2": 4}
Using Tool to calculate: *
Using Tool to calculate: *
Using Tool to calculate: *
I encountered an error while trying to use the tool. This was the error: CalculatorTools.calculate() got an unexpected keyword argument 'num1'.
Tool Make a calculation accepts these inputs: Make a calculation(operation) - Useful to perform any mathematical calculations,
like sum, minus, multiplication, division, etc.
The input to this tool should be a mathematical
expression, a couple examples are `200*7` or `5000/2*10`
To be clear, when commenting out the Ollama configuration, and removing the llm parameter from the Agent defn ... the tool is used properly. (Action and Action Input are then correct).
I'm still learning a lot, and I'm not a pro Python coder ... but I'll try to replicate this issue with just langchain_community.tools and langchain_community.llms and remove CrewAI from the equation.
I’m trying to run my crewAi app with Mixtral:8x7b using ChatOpenAI and ollama
I’m create a docker compose with services of crewAi agents, db and ollama with an network bridge. All the things working fine, but ChatOpenAI don’t working with ollama base api on local :/
anybody have a suggestion to me? Thanks before
This did work at one point, but now it's broken. This is very disappointing.
I'm having the same issue as @hiddenkirby when trying to run the job-posting
example from https://github.com/joaomdmoura/crewAI-examples. My only change is to agents.py
, where I added the lines:
from langchain_community.llms import Ollama
ollama = Ollama(model='llama2')
and the line llm=ollama
to each agent. E.g.:
def research_agent(self):
return Agent(
role='Research Analyst',
goal='Analyze the company website and provided description to extract insights on culture, values, and specific needs.',
tools=[web_search_tool, seper_dev_tool],
backstory='Expert in analyzing company cultures and identifying key values and needs from various sources, including websites and brief descriptions.',
verbose=True,
llm=ollama
)
It seems to load up the local llama2-7b model just fine via ollama, but it repeatedly throws the errors such as the one below.
> Entering new CrewAgentExecutor chain...
Thought: Based on the provided company website and description, I should analyze the content to understand the company's culture, values, and mission. I can leverage this information to attract the right candidates for the job role.
Action: Search in a specific website (search_query: 'crewAI', website: 'https://crewai.com') to gather insights on the company's culture and values.
Action Input: { "website": "https://crewai.com", "query": "crewAI" }
Action 'Search in a specific website (search_query: 'crewAI', website: 'https://crewai.com') to gather insights on the company's culture and values.' don't exist, these are the only available Actions: Search in a specific website: Search in a specific website(search_query: 'string', website: 'string') - A tool that can be used to semantic search a query from a specific URL content.
Search the internet: Search the internet(search_query: 'string') - A tool that can be used to semantic search a query from a txt's content.
Delegate work to co-worker: Delegate work to co-worker(coworker: str, task: str, context: str) - Delegate a specific task to one of the following co-workers: ['Job Description Writer', 'Review and Editing Specialist']
The input to this tool should be the coworker, the task you want them to do, and ALL necessary context to exectue the task, they know nothing about the task, so share absolute everything you know, don't reference things but instead explain them.
Ask question to co-worker: Ask question to co-worker(coworker: str, question: str, context: str) - Ask a specific question to one of the following co-workers: ['Job Description Writer', 'Review and Editing Specialist']
The input to this tool should be the coworker, the question you have for them, and ALL necessary context to ask the question properly, they know nothing about the question, so share absolute everything you know, don't reference things but instead explain them.
After a few iterations, it seems to try to correct itself, but to no avail:
Thank you for correcting me! Here's my revised response:
Thought: I should analyze the company's website and description to understand their culture, values, and mission.
Action: Review the company's website and description to gather insights on their culture and values.
Action Input: N/A
Action 'Review the company's website and description to gather insights on their culture and values.' don't exist, these are the only available Act
ions: Search in a specific website: Search in a specific website(search_query: 'string', website: 'string') - A tool that can be used to semantic search a query from a specific URL content.
Search the internet: Search the internet(search_query: 'string') - A tool that can be used to semantic search a query from a txt's content.
Delegate work to co-worker: Delegate work to co-worker(coworker: str, task: str, context: str) - Delegate a specific task to one of the following co-workers: ['Job Description Writer', 'Review and Editing Specialist']
The input to this tool should be the coworker, the task you want them to do, and ALL necessary context to exectue the task, they know nothing about the task, so share absolute everything you know, don't reference things but instead explain them.
Ask question to co-worker: Ask question to co-worker(coworker: str, question: str, context: str) - Ask a specific question to one of the following co-workers: ['Job Description Writer', 'Review and Editing Specialist']
The input to this tool should be the coworker, the question you have for them, and ALL necessary context to ask the question properly, they know nothing about the question, so share absolute everything you know, don't reference things but instead explain them.
Any help will be much appreciated, as I'd really like to get multi-agents working with local LLMs. Thanks!
I'm having the same issue as @hiddenkirby when trying to run the
job-posting
example from https://github.com/joaomdmoura/crewAI-examples. My only change is toagents.py
, where I added the lines:from langchain_community.llms import Ollama ollama = Ollama(model='llama2')
and the line
llm=ollama
to each agent. E.g.:def research_agent(self): return Agent( role='Research Analyst', goal='Analyze the company website and provided description to extract insights on culture, values, and specific needs.', tools=[web_search_tool, seper_dev_tool], backstory='Expert in analyzing company cultures and identifying key values and needs from various sources, including websites and brief descriptions.', verbose=True, llm=ollama )
It seems to load up the local llama2-7b model just fine via ollama, but it repeatedly throws the errors such as the one below.
> Entering new CrewAgentExecutor chain... Thought: Based on the provided company website and description, I should analyze the content to understand the company's culture, values, and mission. I can leverage this information to attract the right candidates for the job role. Action: Search in a specific website (search_query: 'crewAI', website: 'https://crewai.com') to gather insights on the company's culture and values. Action Input: { "website": "https://crewai.com", "query": "crewAI" } Action 'Search in a specific website (search_query: 'crewAI', website: 'https://crewai.com') to gather insights on the company's culture and values.' don't exist, these are the only available Actions: Search in a specific website: Search in a specific website(search_query: 'string', website: 'string') - A tool that can be used to semantic search a query from a specific URL content. Search the internet: Search the internet(search_query: 'string') - A tool that can be used to semantic search a query from a txt's content. Delegate work to co-worker: Delegate work to co-worker(coworker: str, task: str, context: str) - Delegate a specific task to one of the following co-workers: ['Job Description Writer', 'Review and Editing Specialist'] The input to this tool should be the coworker, the task you want them to do, and ALL necessary context to exectue the task, they know nothing about the task, so share absolute everything you know, don't reference things but instead explain them. Ask question to co-worker: Ask question to co-worker(coworker: str, question: str, context: str) - Ask a specific question to one of the following co-workers: ['Job Description Writer', 'Review and Editing Specialist'] The input to this tool should be the coworker, the question you have for them, and ALL necessary context to ask the question properly, they know nothing about the question, so share absolute everything you know, don't reference things but instead explain them.
After a few iterations, it seems to try to correct itself, but to no avail:
Thank you for correcting me! Here's my revised response: Thought: I should analyze the company's website and description to understand their culture, values, and mission. Action: Review the company's website and description to gather insights on their culture and values. Action Input: N/A Action 'Review the company's website and description to gather insights on their culture and values.' don't exist, these are the only available Act ions: Search in a specific website: Search in a specific website(search_query: 'string', website: 'string') - A tool that can be used to semantic search a query from a specific URL content. Search the internet: Search the internet(search_query: 'string') - A tool that can be used to semantic search a query from a txt's content. Delegate work to co-worker: Delegate work to co-worker(coworker: str, task: str, context: str) - Delegate a specific task to one of the following co-workers: ['Job Description Writer', 'Review and Editing Specialist'] The input to this tool should be the coworker, the task you want them to do, and ALL necessary context to exectue the task, they know nothing about the task, so share absolute everything you know, don't reference things but instead explain them. Ask question to co-worker: Ask question to co-worker(coworker: str, question: str, context: str) - Ask a specific question to one of the following co-workers: ['Job Description Writer', 'Review and Editing Specialist'] The input to this tool should be the coworker, the question you have for them, and ALL necessary context to ask the question properly, they know nothing about the question, so share absolute everything you know, don't reference things but instead explain them.
Any help will be much appreciated, as I'd really like to get multi-agents working with local LLMs. Thanks!
@nfoong , try instantiating your LLM configuration like this instead.
from langchain_openai import ChatOpenAI
llm = ChatOpenAI(
openai_api_base="http://localhost:11434/v1",
openai_api_key="ollama",
model_name="llama2"
)
@hiddenkirby thanks for your reply. Were you able to get it working by doing this?
I just gave that a shot, but the issue still persists, the agent isn't able to choose the right Action as the semantics are wrong:
> Entering new CrewAgentExecutor chain...
Thought: Based on the information provided on the company website and hiring manager's domain, I should analyze the content to understand the company culture, values, and mission. I can use the available tools to gather insights and create a comprehensive report summarizing the findings.
Action: Search in specific website (search_query: 'crewAI culture', website: 'https://crewai.com').
Action Input: None.
Action 'Search in specific website (search_query: 'crewAI culture', website: 'https://crewai.com').' don't exist, these are the only available Actions: Search in a specific website: Search in a specific website(search_query: 'string', website: 'string') - A tool that can be used to semantic search a query from a specific URL content[...]
It's probably meant to be something like this:
Action: Search in specific website(search_query: 'string', website: 'string').
Action Input: {search_query: 'crewAI culture', website: 'https://crewai.com'}
I don't have the expertise or time to debug this further. @kjenney mentioned that it was working before, so maybe a workaround would be to rollback the crewai version until I find a version where this works.
@joaomdmoura any ideas on what's going wrong will be appreciated! Thanks!
I'm getting the same issue if I run any of the demo scripts with a local LLM running on LM Server - it just displays this multiple times:
Thought: What are the latest advancements in AI? Action: [Search the internet]("Recent advancements in AI in 2024") Action Input: {}
Action '[Search the internet]("Recent advancements in AI in 2024")' don't exist, these are the only available Actions: Search the internet: Search the internet(search_query: 'string') - A tool that can be used to semantic search a query from a txt's content.
@nfoong
So, yes and no. Yes, I am able to successfully run scripts with tools against Ollama / Open-sourced LLMs. Here are the two main things I did. I do think there was a CrewAI/CrewAI[tools] update recently as well ... so the water is a bit muddy here (make sure you're running the latest versions of libraries).
from langchain_openai import ChatOpenAI
llm = ChatOpenAI(
openai_api_base="http://localhost:11434/v1",
openai_api_key="ollama",
model_name="mistral_tools"
)
Try to tell the tool to focus on learning tools above all else. (No clue if this is effective or not.) As well as lower the temperature. a. created a modelfile
FROM mistral:latest
TEMPLATE """[INST] {{ .System }} {{ .Prompt }} [/INST]"""
PARAMETER stop "[INST]"
PARAMETER stop "[/INST]"
PARAMETER temperature 0.2
SYSTEM "You are an assistant proficient in understanding and learning how to call functions defined by the user."
b. run ollama create mistral_tools-f ./Modelfile
c. run ollama run mistral_tools
Things are working for me. Occasionally, the LLM will call a function incorrectly as you have observed above, but it tries again and does a better job (eventually). I can move forward with this... somewhat.
@nfoong
So, yes and no. Yes, I am able to successfully run scripts with tools against Ollama / Open-sourced LLMs. Here are the two main things I did. I do think there was a CrewAI/CrewAI[tools] update recently as well ... so the water is a bit muddy here (make sure you're running the latest versions of libraries).
1. Use the OpenAI wrapper instead of Ollama. (the idea is that OpenAI interface better supports Function Calling)
from langchain_openai import ChatOpenAI llm = ChatOpenAI( openai_api_base="http://localhost:11434/v1", openai_api_key="ollama", model_name="mistral_tools" )
2. Try to tell the tool to focus on learning tools above all else. (No clue if this is effective or not.) As well as lower the temperature. a. created a modelfile
FROM mistral:latest TEMPLATE """[INST] {{ .System }} {{ .Prompt }} [/INST]""" PARAMETER stop "[INST]" PARAMETER stop "[/INST]" PARAMETER temperature 0.2 SYSTEM "You are an assistant proficient in understanding and learning how to call functions defined by the user."
b. run
ollama create mistral_tools-f ./Modelfile
c. runollama run mistral_tools
Things are working for me. Occasionally, the LLM will call a function incorrectly as you have observed above, but it tries again and does a better job (eventually). I can move forward with this... somewhat.
Hello. Is there anyupdate on this matter? I tried your above solution but the problem persists.
my_llm= OpenAI(base_url="http://localhost:1234/v1",
api_key="ollama",
model="mistral")
and the example of "action dont exist" (using ChatOpenAi gives the same result):
Action 'Delegate work to Technology Expert' don't exist, these are the only available Actions: Delegate
work to co-worker: Delegate work to co-worker(coworker: str, task: str, context: str) - Delegate a specific task to one of the following co-workers: [Technology Expert, Business Development Consultant]
The input to this tool should be the coworker, the task you want them to do, and ALL necessary context to exectue the task, they know nothing about the task, so share absolute everything you know, don't reference things but instead explain them.
Ask question to co-worker: Ask question to co-worker(coworker: str, question: str, context: str) - Ask a specific question to one of the following co-workers: [Technology Expert, Business Development Consultant]
The input to this tool should be the coworker, the question you have for them, and ALL necessary context to ask the question properly, they know nothing about the question, so share absolute everything you know, don't reference things but instead explain them.
The example above is the startup one from https://github.com/majacinka/crewai-experiments Im using LM Studio with Mistral Instruct. Tested other llm's, very similar results. Is there any tricks to properly run the LM Studio Server? Any link appreciated.
Honestly, I think there are a few pieces to the puzzle here: (I'm still learning)
I gained a good bit of clarity after going through this fella's tutorials on Langchain proper, but I'm still pretty new. ( https://www.youtube.com/playlist?list=PLcQVY5V2UY4Kat6vxC7ESzIIzHWdwlnak )
This did work at one point, but now it's broken. This is very disappointing.
I get it, maybe it's frustrating but this is open source, comments like this one must have that in mind, and maybe a PR that solves it... Or at least something to debug. That would be a constructive, significant contribution to a free, unpaid job that João makes here.
I have tested some of the suggestions, but the only thing that works for me is to use OpenHermes as local model. And yes, I've tried llama3, it doesn't do the work.
FROM openhermes
PARAMETER stop Result
PARAMETER temperature 0.6
PARAMETER num_ctx 8192
It's far from perfect but at least doesn't get stuck at the hidious Action don't exist message.
now Im focusing on task delegation only, without tools. But like @kyuumeitai said, something that is really producing less errors during execution is using 'temperature' values close to zero.
This "seems" to make the LLM follow more accurately the instructions inside crewai code. My only concern is that may limit the "creativity" of the model.
now Im focusing on task delegation only, without tools. But like @kyuumeitai said, something that is really producing less errors during execution is using 'temperature' values close to zero.
This "seems" to make the LLM follow more accurately the instructions inside crewai code. My only concern is that may limit the "creativity" of the model.
Right. That is basically a function of temperature. You might want to configure "creative" agents to use LLMs that don't have low temperature. Not all agents need to use the same LLM.
This issue is stale because it has been open for 30 days with no activity. Remove stale label or comment or this will be closed in 5 days.
This issue was closed because it has been stalled for 5 days with no activity.
I am running the example script from the readme. Other than my API keys, the only change I made was to have the script run off my local LLM.
When I run it, I get a
Action 'Search the internet using the query "Latest Advancements in AI 2024".' don't exist, these are the only available Actions: Search the internet: Search the internet(search_query: 'string') - A tool that can be used to semantic search a query from a txt's content.
.Any assistance would be appreciated.
Code below.
Terminal output (I ctrl + c after the action message)