Ollama and ReadMe Example Documentation

[clarkebishop]

You are doing some very interesting work. Thanks!

A couple of issues with the ReadMe Example.

1. ollama_openhermes = Ollama(model="agent"). "agent" is not a valid Ollama model. Did you mean openhermes here?

2. tools=[] is not well explained. Running Ollama openhermes, I get a lot of errors like, Research current AI trends is not a valid tool, try one of []

This does not happen with an OpenAI key.

I'm guessing that when you use OpenAI, there is a "fallback tool" that takes over. It may be that with Ollama, the developer needs to create some tools. Or something else better explains the problem. That's my question.

This tripped me up when I was looking at your project, so I thought I would contribute by explaining what happened.

ollama_openhermes = Ollama(model="agent")
# Pass Ollama Model to Agents: When creating your agents within the CrewAI framework, you can pass the Ollama model as an argument to the Agent constructor. For instance:

local_expert = Agent(
  role='Local Expert at this city',
  goal='Provide the BEST insights about the selected city',
  backstory="""A knowledgeable local guide with extensive information
  about the city, it's attractions and customs""",
  tools=[
    SearchTools.search_internet,
    BrowserTools.scrape_and_summarize_website,
  ],
  llm=ollama_openhermes, # Ollama model passed here

[MrMWith]

try this... ollama_openhermes = Ollama(model="openhermes")

Or the name of your modelfile version if you did that step.

[stoltzmaniac]

Solving with: https://github.com/joaomdmoura/crewAI/pull/33

[clarkebishop]

I already mad the change from agent to openhermes.

Still, I wanted to point out the gap in the ReadMe documentation.

More important is the behavior regarding tools. It works differently with OpenAI than with Ollama. In fact, without adding tools. the Ollama version does not do much.

[joaomdmoura]

Hey there, so the reason why I had agent in there is because it was a OpenHermes 2.5 I tuned using ollama, I had just create a Modelfile setting up temperature, stop words and other parameters then created it using ollama create, they have some interesting docs on it.

The problem if a smaller model trying to use tools vs GPT4 is because the model is less capable, so it kind assumes it has a tool that it doesn't while GPT4 is better at remembering the tool at its disposal and the format to use them.

Quick question @clarkebishop are you using the new version 0.1.14, I added some improvements to make smaller models work better, there is more coming but it is a step in the right direction. I'll merge #33 updating the README, good catch!

[clarkebishop]

Good point about the custom model file in Ollama @joaomdmoura. Now I understand why it said "agent." I am using Ollama 0.1.17 which is the latest version. You might consider linking to your custom model file.

I love Ollama and it's my go to. I think smaller models are going to be very important going forward. I know Microsoft is doing a lot of work in this area.

I can imagine a project like crewAI using different small llm models for each agent/crew member and even for different tools. 2024 will be exciting!

[prodrammer]

@joaomdmoura I'm not sure if the change from "agent" to "openhermes" is enough. In my crew, using ChatGPT works, but when switching to openhermes, it fails. I've also tried a few other models with similar results. Check out #103. Would your custom model file potentially make things work better?

[LeonidShamis]

Hey there, so the reason why I had agent in there is because it was a OpenHermes 2.5 I tuned using ollama, I had just create a Modelfile setting up temperature, stop words and other parameters then created it using ollama create, they have some interesting docs on it.

The problem if a smaller model trying to use tools vs GPT4 is because the model is less capable, so it kind assumes it has a tool that it doesn't while GPT4 is better at remembering the tool at its disposal and the format to use them.

Quick question @clarkebishop are you using the new version 0.1.14, I added some improvements to make smaller models work better, there is more coming but it is a step in the right direction. I'll merge #33 updating the README, good catch!

Hi @joaomdmoura Could you please share the Modelfile with the modifications to make the Openhermes on Ollama model work better for crewAI Agents? Thank you.