Closed ouening closed 2 weeks ago
You should override Open AI class with Groq stuff like:
import os
from crewai import Agent, Crew, Task
from dotenv import load_dotenv
from langchain_openai import ChatOpenAI
load_dotenv()
llm = ChatOpenAI(
base_url=os.environ.get("GROQ_API_BASE"),
api_key=os.environ.get("GROQ_API_KEY"),
model=os.environ.get("GROQ_MODEL_NAME"),
temperature=0,
)
general_agent = Agent(
role="Math Professor",
goal="""Provide the solution to the students that are asking mathematical questions and give them the answer.""",
backstory="""You are an excellent math professor that likes to solve math questions in a way that everyone can understand your solution""",
allow_delegation=False,
verbose=True,
llm=llm,
)
task = Task(description="""
what is 3 + 5
""", agent=general_agent,
expected_output="8")
crew = Crew(agents=[general_agent], tasks=[task], verbose=2)
result = crew.kickoff()
print(result)
env file:
GROQ_API_BASE=https://api.groq.com/openai/v1
GROQ_MODEL_NAME=llama3-70b-8192
GROQ_API_KEY=your-api-key
more info:
https://console.groq.com/docs/openai https://docs.crewai.com/how-to/LLM-Connections/#ollama-integration
Thanks for your help! Your code runs well. But the code below still encounters problem:
import os
from crewai import Agent, Task, Crew, Process
from crewai_tools import DirectoryReadTool, FileReadTool, WebsiteSearchTool
from crewai_tools import SerperDevTool
from langchain_groq import ChatGroq
from langchain_openai import ChatOpenAI
from dotenv import load_dotenv
search_tool = SerperDevTool()
llm = ChatOpenAI(
model = 'llama3-70b-8192' ,
base_url = "https://api.groq.com/openai/v1",
api_key = 'xxx' ,
temperature=0,
)
print(llm.invoke("what is your knowledge cutoff?")) # for test
topic = 'AI in healthcare'
# Creating a senior researcher agent with memory and verbose mode
researcher = Agent(
role='Senior Researcher',
goal=f'Uncover groundbreaking technologies in {topic}',
verbose=True,
memory=True,
backstory=(
"Driven by curiosity, you're at the forefront of"
"innovation, eager to explore and share knowledge that could change"
"the world."
),
# tools=[search_tool],
allow_delegation=True,
llm=llm
)
writer = Agent(
role='Writer',
goal=f'Narrate compelling tech stories about {topic}',
verbose=True,
memory=True,
backstory=(
"With a flair for simplifying complex topics, you craft"
"engaging narratives that captivate and educate, bringing new"
"discoveries to light in an accessible manner."
),
# tools=[search_tool],
allow_delegation=False,
llm=llm
)
# Creating a writer agent with custom tools and delegation capability
# Research task
research_task = Task(
description=(
f"Identify the next big trend in {topic}."
"Focus on identifying pros and cons and the overall narrative."
"Your final report should clearly articulate the key points,"
"its market opportunities, and potential risks."
),
expected_output='A comprehensive 3 paragraphs long report on the latest AI trends.',
# tools=[search_tool],
agent=researcher,
)
# Writing task with language model configuration
write_task = Task(
description=(
f"Compose an insightful article on {topic}."
"Focus on the latest trends and how it's impacting the industry."
"This article should be easy to understand, engaging, and positive."
),
expected_output=f'A 4 paragraph article on {topic} advancements formatted as markdown.',
# tools=[search_tool],
agent=writer,
async_execution=False,
output_file='new-blog-post.md' # Example of output customization
)
# Forming the tech-focused crew with some enhanced configurations
crew = Crew(
agents=[researcher, writer],
tasks=[research_task, write_task],
process=Process.sequential, # Optional: Sequential task execution is default
memory=True,
cache=True,
max_rpm=100,
share_crew=True
)
# Starting the task execution process with enhanced feedback
result = crew.kickoff()
print(result)
Error message is:
Traceback (most recent call last):
File D:\Apps\WPy64-31131\python-3.11.3.amd64\Lib\site-packages\spyder_kernels\py3compat.py:356 in compat_exec
exec(code, globals, locals)
File d:\mega\misc\python\crewai\crewai_test.py:122
result = crew.kickoff()
File D:\Apps\WPy64-31131\python-3.11.3.amd64\Lib\site-packages\crewai\crew.py:252 in kickoff
result = self._run_sequential_process()
File D:\Apps\WPy64-31131\python-3.11.3.amd64\Lib\site-packages\crewai\crew.py:293 in _run_sequential_process
output = task.execute(context=task_output)
File D:\Apps\WPy64-31131\python-3.11.3.amd64\Lib\site-packages\crewai\task.py:173 in execute
result = self._execute(
File D:\Apps\WPy64-31131\python-3.11.3.amd64\Lib\site-packages\crewai\task.py:182 in _execute
result = agent.execute_task(
File D:\Apps\WPy64-31131\python-3.11.3.amd64\Lib\site-packages\crewai\agent.py:207 in execute_task
memory = contextual_memory.build_context_for_task(task, context)
File D:\Apps\WPy64-31131\python-3.11.3.amd64\Lib\site-packages\crewai\memory\contextual\contextual_memory.py:22 in build_context_for_task
context.append(self._fetch_stm_context(query))
File D:\Apps\WPy64-31131\python-3.11.3.amd64\Lib\site-packages\crewai\memory\contextual\contextual_memory.py:31 in _fetch_stm_context
stm_results = self.stm.search(query)
File D:\Apps\WPy64-31131\python-3.11.3.amd64\Lib\site-packages\crewai\memory\short_term\short_term_memory.py:23 in search
return self.storage.search(query=query, score_threshold=score_threshold)
File D:\Apps\WPy64-31131\python-3.11.3.amd64\Lib\site-packages\crewai\memory\storage\rag_storage.py:90 in search
else self.app.search(query, limit)
File D:\Apps\WPy64-31131\python-3.11.3.amd64\Lib\site-packages\embedchain\embedchain.py:635 in search
return [{"context": c[0], "metadata": c[1]} for c in self.db.query(**params)]
File D:\Apps\WPy64-31131\python-3.11.3.amd64\Lib\site-packages\embedchain\vectordb\chroma.py:220 in query
result = self.collection.query(
File D:\Apps\WPy64-31131\python-3.11.3.amd64\Lib\site-packages\chromadb\api\models\Collection.py:327 in query
valid_query_embeddings = self._embed(input=valid_query_texts)
File D:\Apps\WPy64-31131\python-3.11.3.amd64\Lib\site-packages\chromadb\api\models\Collection.py:633 in _embed
return self._embedding_function(input=input)
File D:\Apps\WPy64-31131\python-3.11.3.amd64\Lib\site-packages\chromadb\api\types.py:193 in __call__
result = call(self, input)
File D:\Apps\WPy64-31131\python-3.11.3.amd64\Lib\site-packages\chromadb\utils\embedding_functions.py:188 in __call__
embeddings = self._client.create(
File D:\Apps\WPy64-31131\python-3.11.3.amd64\Lib\site-packages\openai\resources\embeddings.py:113 in create
return self._post(
File D:\Apps\WPy64-31131\python-3.11.3.amd64\Lib\site-packages\openai\_base_client.py:1208 in post
return cast(ResponseT, self.request(cast_to, opts, stream=stream, stream_cls=stream_cls))
File D:\Apps\WPy64-31131\python-3.11.3.amd64\Lib\site-packages\openai\_base_client.py:897 in request
return self._request(
File D:\Apps\WPy64-31131\python-3.11.3.amd64\Lib\site-packages\openai\_base_client.py:988 in _request
raise self._make_status_error_from_response(err.response) from None
NotFoundError: Error code: 404 - {'error': {'message': 'Unknown request URL: POST /openai/v1/embeddings. Please check the URL for typos, or see the docs at https://console.groq.com/docs/', 'type': 'invalid_request_error', 'code': 'unknown_url'}}
I can't figure out what is the problem in my code.
https://docs.crewai.com/core-concepts/Memory/#how-memory-systems-empower-agents - read this, you are using a non openai endpoint. by default memory uses Openai - you need to specify an embedding
It seems that Ollama's embedding, such as nomic-embed-text, is not currently supported. I tried using other free APIs, such as Gemini, but they all failed, and detailed documentation on the embedder is still lacking.
correct, Embedchain is used for RAG / Memory and they do not currently support Ollama so that an issue that cannot be resolved by crewai. there is a PR for them to implement - https://github.com/embedchain/embedchain/pull/1344
Thank you for your reply, I will continue to follow up.
Looks like it was just merged in and released... So hopefully soon we can have local embedding support. Which embedding model would we recommend as the default for ollama?
Yup, I'll make sure we update the dependency! :D
This issue is stale because it has been open for 30 days with no activity. Remove stale label or comment or this will be closed in 5 days.
This issue was closed because it has been stalled for 5 days with no activity.
It seems that the embedding did not work? How could I use different embedding model (like some models in ollama) instead of openai?