Closed binhphamthanh closed 4 months ago
1: Ensure ollama is installed and running, and is version 0.1.27 or higher:
ps aux | grep ollama
ollama 1810 55.4 2.5 291658704 1663360 ? Ssl 12:56 81:16 /usr/local/bin/ollama serve
ollama -v
ollama version is 0.1.27
2: Get the ollama model name
ollama list
NAME ID SIZE MODIFIED
mistral:7b-instruct-v0.2-q8_0 3f321fd2a1c3 7.7 GB 5 days ago
3: At top of your python code, before the crewai imports:
os.environ['OPENAI_API_BASE'] = 'http://localhost:11434/v1'
os.environ['OPENAI_MODEL_NAME'] = 'mistral:7b-instruct-v0.2-q8_0'
os.environ['OPENAI_API_KEY'] = 'NA'
This should work on crewai 0.19
1: Ensure ollama is installed and running, and is version 0.1.27 or higher:
ps aux | grep ollama ollama 1810 55.4 2.5 291658704 1663360 ? Ssl 12:56 81:16 /usr/local/bin/ollama serve ollama -v ollama version is 0.1.27
2: Get the ollama model name
ollama list NAME ID SIZE MODIFIED mistral:7b-instruct-v0.2-q8_0 3f321fd2a1c3 7.7 GB 5 days ago
3: At top of your python code, before the crewai imports:
os.environ['OPENAI_API_BASE'] = 'http://localhost:11434/v1' os.environ['OPENAI_MODEL_NAME'] = 'mistral:7b-instruct-v0.2-q8_0' os.environ['OPENAI_API_KEY'] = 'NA'
This should work on crewai 0.19
ollama run llama2
(llama2:latest)import os
from crewai import Agent, Task, Crew, Process
from crewai_tools import SerperDevTool
from langchain_community.llms import Ollama
# You can choose to use a local model through Ollama for example. See https://docs.crewai.com/how-to/LLM-Connections/ for more information.
os.environ["OPENAI_API_BASE"] = 'http://localhost:11434/v1'
os.environ["OPENAI_MODEL_NAME"] ='llama2:latest' # Adjust based on available model
os.environ["OPENAI_API_KEY"] ='sk-111111111111111111111111111111111111111111111111'
llama2 = Ollama(model="llama2")
search_tool = SerperDevTool()
# Define your agents with roles and goals
researcher = Agent(
role='Senior Research Analyst',
goal='Uncover cutting-edge developments in AI and data science',
backstory="""You work at a leading tech think tank.
Your expertise lies in identifying emerging trends.
You have a knack for dissecting complex data and presenting actionable insights.""",
verbose=True,
allow_delegation=False,
tools=[search_tool],
llm=llama2
)
writer = Agent(
role='Tech Content Strategist',
goal='Craft compelling content on tech advancements',
backstory="""You are a renowned Content Strategist, known for your insightful and engaging articles.
You transform complex concepts into compelling narratives.""",
verbose=True,
allow_delegation=True,
llm=llama2
)
# Create tasks for your agents
task1 = Task(
description="""Conduct a comprehensive analysis of the latest advancements in AI in 2024.
Identify key trends, breakthrough technologies, and potential industry impacts.""",
expected_output="Full analysis report in bullet points",
agent=researcher
)
task2 = Task(
description="""Using the insights provided, develop an engaging blog
post that highlights the most significant AI advancements.
Your post should be informative yet accessible, catering to a tech-savvy audience.
Make it sound cool, avoid complex words so it doesn't sound like AI.""",
expected_output="Full blog post of at least 4 paragraphs",
agent=writer
)
# Instantiate your crew with a sequential process
crew = Crew(
agents=[researcher, writer],
tasks=[task1, task2],
verbose=2, # You can set it to 1 or 2 to different logging levels
)
# Get your crew to work!
result = crew.kickoff()
print("######################")
print(result)
I get
(.venv) me@my-mbp-2 crewai_tests % python [crewai_test.py](http://crewai_test.py/)
[DEBUG]: == Working Agent: Senior Research Analyst
[INFO]: == Starting Task: Conduct a comprehensive analysis of the latest advancements in AI in 2024.
Identify key trends, breakthrough technologies, and potential industry impacts.
> Entering new CrewAgentExecutor chain...
Thought: I understand the task at hand and will use the available tools to conduct a comprehensive analysis of the latest advancements in AI in 2024.
Action: Search the internet for "AI trends 2024"
Action Input: {"query": "AI trends 2024"}
Action 'Search the internet for "AI trends 2024"' don't exist, these are the only available Actions: Search the internet: Search the internet(search_query: 'string') - A tool that can be used to semantic search a query from a txt's content.
Thought: I understand the task at hand and will use the available tools to conduct a comprehensive analysis of the latest advancements in AI in 2024.
Action: Search the internet for "AI trends 2024"
Action Input: {"query": "AI trends 2024"}
Action 'Search the internet for "AI trends 2024"' don't exist, these are the only available Actions: Search the internet: Search the internet(search_query: 'string') - A tool that can be used to semantic search a query from a txt's content.
It's not understanding how to use tools... it seems.
If i:
os.environ["OPENAI_API_BASE"] = 'http://localhost:11434/v1'
os.environ["OPENAI_MODEL_NAME"] ='llama2:latest' # Adjust based on available model
os.environ["OPENAI_API_KEY"] ='sk-111111111111111111111111111111111111111111111111'
llama2 = Ollama(model="llama2")
The search function (and subsequently the script) will work. If I remove the search_tool from the configuration, interacting with the local model does work.
When using local models you must set OPENAI_API_KEY=NA
@lhermoso I haven't seen this:
When using local models you must set OPENAI_API_KEY=NA
Documented anywhere, nor have I been able to find this in the code anywhere. Can you explain where you found this and how you knew to do this?
For context, doing this has not improved my situation of running locally either..
@lhermoso I haven't seen this:
When using local models you must set OPENAI_API_KEY=NA
Documented anywhere, nor have I been able to find this in the code anywhere. Can you explain where you found this and how you knew to do this?
For context, doing this has not improved my situation of running locally either..
If you go to https://docs.crewai.com/how-to/LLM-Connections/#from-huggingfacehub-endpoint you will find this:
@lhermoso
Ah, the Ollama integration section is what I needed, as I'm using Ollama.
Thanks for bringing this part of the docs to my attention :)
We also need better docs, sorry about that, it's on our radas
"_It seems we encountered an unexpected error while trying to use the tool. This was the error: 'OPENAI_APIKEY'" I got this message while trying to use tools. Can you please let me know how to switch into local LLM models.