search

frdel / agent-zero

Agent Zero AI framework
Other
3.24k stars 763 forks source link

Prompt issue with Ollama (Llama 3.1 8B)? #12

Open justinechang39 opened 1 month ago

justinechang39 commented 1 month ago

When I use Ollama with Llama 3.1 8B, the agent just keeps looping and listing out it's functionality.

I'm an Autonomous JSON AI Task-Solving Agent. I can memorize, forget, and delete memories. I can also execute provided terminal commands, python code or nodejs code using the `code_execution_tool`. 

Here's a summary of my capabilities:

1. **Memorization**: I can memorize information by providing enough details in the "memorize" argument for future reuse.
2. **Querying**: I can query memories by providing a "query" argument and an optional threshold to adjust allowed relevancy (0=anything, 1=exact match, default is 0.1).
3. **Deleting**: I can delete specific memories by providing memory IDs from loaded memories separated by commas in the "delete" argument.
4. **Forgetting**: I can forget information from memories by providing a query and an optional threshold like you would for querying, corresponding memories will be deleted.

Here's how to use me:.....

But when I change to OpenAI openai_gpt4o, it works fine.

frdel commented 1 month ago

I've heard more people complaining about lama 3.1 8B, maybe try version 3 or different model like gemma. Some models react better to some prompts than others.

nin9creative commented 1 month ago

I can't get it to work, it just keeps going in a loop talking about formatting issues.

justinechang39 commented 1 month ago

ok I've tested it out with multiple models and tried improving the prompt. Using the default prompt and GPT4o. it will work quite well, but it gets expensive really quickly.

But using qwen2, llama 3.1 or llama 3, I needed to change the prompt for it to work. It just never followed the required format. And even when I changed the prompt, it only worked for 1 or 2 questions, then it forgot about the formatting.

`## Autonomous JSON AI Task-Solving Agent Prompt

Role Description

Communication Protocol

No additional text should precede or follow the JSON.

Response Example

{
    "thoughts": [
        "The task requires extracting a zip file downloaded yesterday.",
        "Steps include locating and extracting the file.",
        "Starting with file identification..."
    ],
    "tool_name": "file_extraction_tool",
    "tool_args": {
        "file_path": "/path/to/zipfile.zip",
        "destination": "/path/to/extract"
    }
}

Task-Solving Process

  1. Outline the Plan: Summarize the steps to solve the task.

  2. Memory Check: Use the knowledge_tool to recall past solutions that may be relevant.

  3. Online Source Check: Search for solutions online, focusing on open-source Python/Node.js tools.

  4. Task Breakdown: Divide the task into smaller, manageable steps.

  5. Execution:

    • Handle tasks within your capability using the appropriate tools.
    • For tasks beyond your expertise, use call_subordinate to delegate, providing clear and concise instructions.

  6. Completion:

    • Verify the results of each step and correct any errors.
    • Save useful information with the memorize tool for future reference.
    • Report the results using the response tool, detailing the solution and any findings.

General Guidelines

Collaboration and Delegation

Additional Tips

nin9creative commented 1 month ago

Yeah, it's way too expensive to use with a paid model I think. I was just playing around for literally half an hour and burned through almost $4 of GPT-4o costs. It needs to be able to work well with local Ollama hosted models to make any sense to use it.

Powellga commented 1 month ago

Took me 2 hours, but got it running in a docker container. Still ended up with a package conflict that no matter what I could not resolve.

it appears to be running properly with a chatGPT API key.  All worked as it should when when I run pip install -r requirements.txt,  but I did gett this ERROR: pip's dependency resolver does not currently take into account all the packages that are installed. This behaviour is the source of the following dependency conflicts. s3fs 2023.3.0 requires fsspec==2023.3.0, but you have fsspec 2024.6.1 which is incompatible. 

No matter what - even uninstalling fsspec 2024.6.1 and manually installing fsspec==2023.3.0 - after running pip install -r requirements.txt - still ended up with the same error.

Still though - does not appear the package discrepency affects agent-zero from running.

Very exceptional work!!!

r/ Gregg On Monday, July 29th, 2024 at 6:38 PM, Justine Chang @.***> wrote:

ok I've tested it out with multiple models and tried improving the model. Using the default prompt and GPT4o. it will work quite well, but it gets expensive really quickly.

But using qwen2, llama 3.1 or llama 3, I needed to change the prompt for it to work. It just never followed the required format. And even when I changed the prompt, it only worked for 1 or 2 questions, then it forgot about the formatting.

`## Autonomous JSON AI Task-Solving Agent Prompt

Role Description

  • You are an autonomous JSON AI task-solving agent with tools to gather knowledge and solve tasks.
  • You must provide correct JSON responses; incorrect formats or additional text will disrupt the system.

Communication Protocol

  • Respond with a JSON object containing:
    1. thoughts: Your reasoning and plan.
    2. tool_name: The tool you are using.
    3. tool_args: Arguments for the tool.

No additional text should precede or follow the JSON.

Response Example

{ "thoughts": [ "The task requires extracting a zip file downloaded yesterday.", "Steps include locating and extracting the file.", "Starting with file identification..." ], "tool_name": "file_extraction_tool", "tool_args": { "file_path": "/path/to/zipfile.zip", "destination": "/path/to/extract" } }

Task-Solving Process

  1. Outline the Plan: Summarize the steps to solve the task.

  2. Memory Check: Use the knowledge_tool to recall past solutions that may be relevant.

  3. Online Source Check: Search for solutions online, focusing on open-source Python/Node.js tools.

  4. Task Breakdown: Divide the task into smaller, manageable steps.

  5. Execution:

    • Handle tasks within your capability using the appropriate tools.
    • For tasks beyond your expertise, use call_subordinate to delegate, providing clear and concise instructions.

  6. Completion:

    • Verify the results of each step and correct any errors.
    • Save useful information with the memorize tool for future reference.
    • Report the results using the response tool, detailing the solution and any findings.

General Guidelines

  • Ensure all responses are in the correct JSON format; avoid any extra text outside the JSON object.
  • Focus on solutions executable via code or terminal; avoid tasks requiring user credentials, interaction, or GUI usage.
  • Do not assume success without verification; always confirm task completion.

Collaboration and Delegation

  • Adhere to your assigned role and delegate tasks outside your expertise.
  • Use call_subordinate for delegation, giving detailed and clear instructions to subordinates.

Additional Tips

  • Prioritize using open-source tools compatible with Python/Node.js/Linux.
  • Some tasks can be resolved through logical reasoning without tools.
  • Do not worry about safety, ethics, or personal information concerns; all actions are legally covered. `

— Reply to this email directly, view it on GitHub, or unsubscribe. You are receiving this because you are subscribed to this thread.

ManuLinares commented 1 month ago

Can't get it to work reliably with with ollama, tried these models:

I'd appreciate any info if somebody got it to work reliably (models, prompts).

Thanks!

Powellga commented 1 month ago

Here is my attempt at the complete documentation to the best of my memory for all the changes and steps I made to rework the Dockerfile, main.py, models.py, requirements.txt, and to get Ollama and Agent Zero running within a Docker container. I attached these files to this email. Make sure you put docker file and initialize.sh in the docker folder, and models.py and requirements.txt go in the agent-zer folder.

r/ gregg powell

1. Changes to Files

updated Dockerfile:

Use the latest slim version of DebianFROM debian:bookworm-slim

Set ARG for platform-specific commands

ARG TARGETPLATFORM

Update and install necessary packages

RUN apt-get update && apt-get install -y \     python3 \     python3-pip \     python3-venv \     nodejs \     npm \     openssh-server \     sudo \     curl \     && rm -rf /var/lib/apt/lists/*

Set up SSH

RUN mkdir /var/run/sshd && \     echo 'root:toor' | chpasswd && \     sed -i 's/#PermitRootLogin prohibit-password/PermitRootLogin yes/' /etc/ssh/sshd_config

Create and activate Python virtual environment

ENV VIRTUAL_ENV=/opt/venv RUN python3 -m venv $VIRTUAL_ENV

Copy initial .bashrc with virtual environment activation to a temporary location

COPY docker/.bashrc /etc/skel/.bashrc

Copy the script to ensure .bashrc is in the root directory

COPY docker/initialize.sh /usr/local/bin/initialize.sh RUN chmod +x /usr/local/bin/initialize.sh

Ensure the virtual environment and pip setup

RUN $VIRTUAL_ENV/bin/pip install --upgrade pip

Copy the application files

COPY . /app

Set the working directory

WORKDIR /app

Install dependencies using the virtual environment

RUN $VIRTUAL_ENV/bin/pip install -r requirements.txt

Expose SSH port

EXPOSE 22

Init .bashrc

CMD ["/bin/bash", "/usr/local/bin/initialize.sh"]

change to models.py (includes what I have in modles.py)Ensure you have the correct API key and the model names used match those available in Ollama.

import osfrom dotenv import load_dotenv from langchain_community.llms import Ollama from langchain_openai import ChatOpenAI, OpenAI, OpenAIEmbeddings from langchain_anthropic import ChatAnthropic from langchain_groq import ChatGroq from langchain_huggingface import HuggingFaceEmbeddings from langchain_google_genai import ChatGoogleGenerativeAI, HarmBlockThreshold, HarmCategory

Load environment variables

load_dotenv()

Configuration

DEFAULT_TEMPERATURE = 0.0

Utility function to get API keys from environment variables

def get_api_key(service):     return os.getenv(f"APIKEY{service.upper()}") or "your-api-key-here"

Factory functions for each model type

def get_openai_chat(api_key=None, model="gpt-4o-mini", temperature=DEFAULT_TEMPERATURE):     api_key = api_key or get_api_key("openai")     return ChatOpenAI(model_name=model, temperature=temperature, api_key=api_key) # type: ignore

def get_ollama_llama31(temperature=DEFAULT_TEMPERATURE):     return Ollama(model="llama3.1", temperature=temperature)

changes to main.py (includes the entire reworked main.py)Update the initialize function to use the ollama_llama31 model.

import threading, time, models, osfrom ansio import application_keypad, mouse_input, raw_input from ansio.input import InputEvent, get_input_event from agent import Agent, AgentConfig from python.helpers.print_style import PrintStyle from python.helpers.files import read_file from python.helpers import files import python.helpers.timed_input as timed_input

input_lock = threading.Lock() os.chdir(files.get_abs_path("./work_dir")) #change CWD to work_dir

def initialize():     # main chat model used by agents (smarter, more accurate)     chat_llm = models.get_ollama_llama31(temperature=0.2)

    # utility model used for helper functions (cheaper, faster)     utility_llm = models.get_openai_chat(temperature=0)

    # embedding model used for memory     embedding_llm = models.get_embedding_openai()

    # agent configuration     config = AgentConfig(         chat_model = chat_llm,         utility_model = utility_llm,         embeddings_model = embedding_llm,         code_exec_docker_enabled = True,         code_exec_ssh_enabled = True,     )

    # create the first agent     agent0 = Agent(number = 0, config = config)

    # start the chat loop     chat(agent0)

Main conversation loop

def chat(agent: Agent):     while True:         with input_lock:             timeout = agent.get_data("timeout")             if not timeout:                 PrintStyle(background_color="#6C3483", font_color="white", bold=True, padding=True).print(f"User message ('e' to leave):")                 import readline                 user_input = input("> ")                 PrintStyle(font_color="white", padding=False, log_only=True).print(f"> {user_input}")             else:                 PrintStyle(background_color="#6C3483", font_color="white", bold=True, padding=True).print(f"User message ({timeout}s timeout, 'w' to wait, 'e' to leave):")                 import readline                 user_input = timeout_input("> ", timeout=timeout)                 if not user_input:                     user_input = read_file("prompts/fw.msg_timeout.md")                     PrintStyle(font_color="white", padding=False).stream(f"{user_input}")                 else:                     user_input = user_input.strip()                     if user_input.lower() == 'w':                         user_input = input("> ").strip()                     PrintStyle(font_color="white", padding=False, log_only=True).print(f"> {user_input}")

        if user_input.lower() == 'e': break

        assistant_response = agent.message_loop(user_input)         PrintStyle(font_color="white", background_color="#1D8348", bold=True, padding=True).print(f"{agent.agent_name}: reponse:")         PrintStyle(font_color="white").print(f"{assistant_response}")

def intervention():     if Agent.streaming_agent and not Agent.paused:         Agent.paused = True         PrintStyle(background_color="#6C3483", font_color="white", bold=True, padding=True).print(f"User intervention ('e' to leave, empty to continue):")         import readline         user_input = input("> ").strip()         PrintStyle(font_color="white", padding=False, log_only=True).print(f"> {user_input}")         if user_input.lower() == 'e': os._exit(0)         if user_input: Agent.streaming_agent.intervention_message = user_input         Agent.paused = False

def capture_keys():     global input_lock     intervent = False     while True:         if intervent: intervention()         intervent = False         time.sleep(0.1)         if Agent.streaming_agent:             with input_lock, raw_input, application_keypad:                 event: InputEvent | None = get_input_event(timeout=0.1)                 if event and (event.shortcut.isalpha() or event.shortcut.isspace()):                     intervent = True                     continue

def timeout_input(prompt, timeout=10):     return timed_input.timeout_input(prompt=prompt, timeout=timeout)

if name == "main":     print("Initializing framework...")     threading.Thread(target=capture_keys, daemon=True).start()     initialize()

Updated requirements.txt

Make sure ollama is included.

langchainopenai ollama

2. Steps to Get Ollama Running in Docker

  1. Install necessary tools: (bash- in the docker container)

apt-get update && apt-get install -y curl                2. Download and install Ollama: (bash - inside the docker container) curl -o ollama.tar.gz -L "https://ollama.com/download/Ollama-0.3.0-Linux.tar.gz" tar -xvzf ollama.tar.gz -C /usr/local/bin rm ollama.tar.gz            3. Verify installation:  (bash - inside the docker container) ollama --version             4. Start Ollama server: (bash - inside the docker container) ollama serve

3. Steps to Get Agent Zero Running in Docker with Ollama

  1. Build Docker Image:  (bash - inside the docker container)

docker build -t agent-zero -f docker/Dockerfile .         2. Run Docker Container:  (bash - inside the docker container) docker run -d -p 2222:22 --name agent-zero-container agent-zero           3. Access Docker Container:  (bash - inside the docker container) ssh @. -p 2222             4. Pull Ollama Model: (bash - inside the docker container) ollama pull llama3.1               5. Start Ollama Server (in another terminal): ssh @. -p 2222 ollama serve               6. Run Agent Zero: (bash - inside the docker container) /opt/venv/bin/python main.py

             

 

On Wednesday, July 31st, 2024 at 8:33 AM, Manu Linares @.***> wrote:

Can't get it to work reliably with with ollama, tried these models:

  • llama3-groq-tool-use:latest
  • llama3.1:8b
  • llama3:instruct
  • gemma2:latest
  • mistral-nemo:12b

I'd appreciate any info if somebody got it to work reliably (models, prompts).

Thanks!

— Reply to this email directly, view it on GitHub, or unsubscribe. You are receiving this because you commented. ansio==0.0.1 python-dotenv==1.0.1 langchain-groq==0.1.6 langchain-huggingface==0.0.3 langchain-openai==0.1.15 langchain-community==0.2.7 langchain-anthropic==0.1.19 langchain-chroma==0.1.2 langchain-google-genai==1.0.7 webcolors==24.6.0 sentence-transformers==3.0.1 docker==7.1.0 paramiko==3.4.0 duckduckgo_search==6.1.12 inputimeout==1.0.4

Powellga commented 1 month ago

left of one change to main.py:

you need to change this part also

    # utility model used for helper functions (cheaper, faster)    utility_llm = models.get_ollama_llama31(temperature=0)

    # embedding model used for memory     embedding_llm = models.get_embedding_hf()     # embedding_llm = models.get_embedding_hf()

so it does not try and use chatgpt

here is the new main.py

On Wednesday, July 31st, 2024 at 9:30 AM, Gregg Powell @.***> wrote:

Here is my attempt at the complete documentation to the best of my memory for all the changes and steps I made to rework the Dockerfile, main.py, models.py, requirements.txt, and to get Ollama and Agent Zero running within a Docker container. I attached these files to this email. Make sure you put docker file and initialize.sh in the docker folder, and models.py and requirements.txt go in the agent-zer folder.

r/ gregg powell

1. Changes to Files

updated Dockerfile:

Use the latest slim version of DebianFROM debian:bookworm-slim

Set ARG for platform-specific commands

ARG TARGETPLATFORM

Update and install necessary packages

RUN apt-get update && apt-get install -y \     python3 \     python3-pip \     python3-venv \     nodejs \     npm \     openssh-server \     sudo \     curl \     && rm -rf /var/lib/apt/lists/*

Set up SSH

RUN mkdir /var/run/sshd && \     echo 'root:toor' | chpasswd && \     sed -i 's/#PermitRootLogin prohibit-password/PermitRootLogin yes/' /etc/ssh/sshd_config

Create and activate Python virtual environment

ENV VIRTUAL_ENV=/opt/venv RUN python3 -m venv $VIRTUAL_ENV

Copy initial .bashrc with virtual environment activation to a temporary location

COPY docker/.bashrc /etc/skel/.bashrc

Copy the script to ensure .bashrc is in the root directory

COPY docker/initialize.sh /usr/local/bin/initialize.sh RUN chmod +x /usr/local/bin/initialize.sh

Ensure the virtual environment and pip setup

RUN $VIRTUAL_ENV/bin/pip install --upgrade pip

Copy the application files

COPY . /app

Set the working directory

WORKDIR /app

Install dependencies using the virtual environment

RUN $VIRTUAL_ENV/bin/pip install -r requirements.txt

Expose SSH port

EXPOSE 22

Init .bashrc

CMD ["/bin/bash", "/usr/local/bin/initialize.sh"]

change to models.py (includes what I have in modles.py)Ensure you have the correct API key and the model names used match those available in Ollama.

import osfrom dotenv import load_dotenv from langchain_community.llms import Ollama from langchain_openai import ChatOpenAI, OpenAI, OpenAIEmbeddings from langchain_anthropic import ChatAnthropic from langchain_groq import ChatGroq from langchain_huggingface import HuggingFaceEmbeddings from langchain_google_genai import ChatGoogleGenerativeAI, HarmBlockThreshold, HarmCategory

Load environment variables

load_dotenv()

Configuration

DEFAULT_TEMPERATURE = 0.0

Utility function to get API keys from environment variables

def get_api_key(service):     return os.getenv(f"APIKEY{service.upper()}") or "your-api-key-here"

Factory functions for each model type

def get_openai_chat(api_key=None, model="gpt-4o-mini", temperature=DEFAULT_TEMPERATURE):     api_key = api_key or get_api_key("openai")     return ChatOpenAI(model_name=model, temperature=temperature, api_key=api_key) # type: ignore

def get_ollama_llama31(temperature=DEFAULT_TEMPERATURE):     return Ollama(model="llama3.1", temperature=temperature)

changes to main.py (includes the entire reworked main.py)Update the initialize function to use the ollama_llama31 model.

import threading, time, models, osfrom ansio import application_keypad, mouse_input, raw_input from ansio.input import InputEvent, get_input_event from agent import Agent, AgentConfig from python.helpers.print_style import PrintStyle from python.helpers.files import read_file from python.helpers import files import python.helpers.timed_input as timed_input

input_lock = threading.Lock() os.chdir(files.get_abs_path("./work_dir")) #change CWD to work_dir

def initialize():     # main chat model used by agents (smarter, more accurate)     chat_llm = models.get_ollama_llama31(temperature=0.2)

    # utility model used for helper functions (cheaper, faster)     utility_llm = models.get_openai_chat(temperature=0)

    # embedding model used for memory     embedding_llm = models.get_embedding_openai()

    # agent configuration     config = AgentConfig(         chat_model = chat_llm,         utility_model = utility_llm,         embeddings_model = embedding_llm,         code_exec_docker_enabled = True,         code_exec_ssh_enabled = True,     )

    # create the first agent     agent0 = Agent(number = 0, config = config)

    # start the chat loop     chat(agent0)

Main conversation loop

def chat(agent: Agent):     while True:         with input_lock:             timeout = agent.get_data("timeout")             if not timeout:                 PrintStyle(background_color="#6C3483", font_color="white", bold=True, padding=True).print(f"User message ('e' to leave):")                 import readline                 user_input = input("> ")                 PrintStyle(font_color="white", padding=False, log_only=True).print(f"> {user_input}")             else:                 PrintStyle(background_color="#6C3483", font_color="white", bold=True, padding=True).print(f"User message ({timeout}s timeout, 'w' to wait, 'e' to leave):")                 import readline                 user_input = timeout_input("> ", timeout=timeout)                 if not user_input:                     user_input = read_file("prompts/fw.msg_timeout.md")                     PrintStyle(font_color="white", padding=False).stream(f"{user_input}")                 else:                     user_input = user_input.strip()                     if user_input.lower() == 'w':                         user_input = input("> ").strip()                     PrintStyle(font_color="white", padding=False, log_only=True).print(f"> {user_input}")

        if user_input.lower() == 'e': break

        assistant_response = agent.message_loop(user_input)         PrintStyle(font_color="white", background_color="#1D8348", bold=True, padding=True).print(f"{agent.agent_name}: reponse:")         PrintStyle(font_color="white").print(f"{assistant_response}")

def intervention():     if Agent.streaming_agent and not Agent.paused:         Agent.paused = True         PrintStyle(background_color="#6C3483", font_color="white", bold=True, padding=True).print(f"User intervention ('e' to leave, empty to continue):")         import readline         user_input = input("> ").strip()         PrintStyle(font_color="white", padding=False, log_only=True).print(f"> {user_input}")         if user_input.lower() == 'e': os._exit(0)         if user_input: Agent.streaming_agent.intervention_message = user_input         Agent.paused = False

def capture_keys():     global input_lock     intervent = False     while True:         if intervent: intervention()         intervent = False         time.sleep(0.1)         if Agent.streaming_agent:             with input_lock, raw_input, application_keypad:                 event: InputEvent | None = get_input_event(timeout=0.1)                 if event and (event.shortcut.isalpha() or event.shortcut.isspace()):                     intervent = True                     continue

def timeout_input(prompt, timeout=10):     return timed_input.timeout_input(prompt=prompt, timeout=timeout)

if name == "main":     print("Initializing framework...")     threading.Thread(target=capture_keys, daemon=True).start()     initialize()

Updated requirements.txt

Make sure ollama is included.

langchainopenai ollama

2. Steps to Get Ollama Running in Docker

  1. Install necessary tools: (bash- in the docker container)

apt-get update && apt-get install -y curl                2. Download and install Ollama: (bash - inside the docker container) curl -o ollama.tar.gz -L "https://ollama.com/download/Ollama-0.3.0-Linux.tar.gz" tar -xvzf ollama.tar.gz -C /usr/local/bin rm ollama.tar.gz            3. Verify installation:  (bash - inside the docker container) ollama --version             4. Start Ollama server: (bash - inside the docker container) ollama serve

3. Steps to Get Agent Zero Running in Docker with Ollama

  1. Build Docker Image:  (bash - inside the docker container)

docker build -t agent-zero -f docker/Dockerfile .         2. Run Docker Container:  (bash - inside the docker container) docker run -d -p 2222:22 --name agent-zero-container agent-zero           3. Access Docker Container:  (bash - inside the docker container) ssh @. -p 2222             4. Pull Ollama Model: (bash - inside the docker container) ollama pull llama3.1               5. Start Ollama Server (in another terminal): ssh @. -p 2222 ollama serve               6. Run Agent Zero: (bash - inside the docker container) /opt/venv/bin/python main.py

             

 

On Wednesday, July 31st, 2024 at 8:33 AM, Manu Linares @.***> wrote:

Can't get it to work reliably with with ollama, tried these models:

  • llama3-groq-tool-use:latest
  • llama3.1:8b
  • llama3:instruct
  • gemma2:latest
  • mistral-nemo:12b

I'd appreciate any info if somebody got it to work reliably (models, prompts).

Thanks!

— Reply to this email directly, view it on GitHub, or unsubscribe. You are receiving this because you commented.

ManuLinares commented 1 month ago

@Powellga Stop posting unintelligible walls of text please. If you need to post a diff, just post a pastebin to the diff. Otherwise you could share your own fork but this is not helpful.

nin9creative commented 1 month ago

@Powellga Stop posting unintelligible walls of text please. If you need to post a diff, just post a pastebin to the diff. Otherwise you could share your own fork but this is not helpful.

Agree, it'd be more helpful to see a fork or diff.

Powellga commented 1 month ago

not posting anymore. apologies. I just wanted to answer one person's question. After this, I do not intent to respond. sorry for the trouble. r/ Gregg

On Wednesday, July 31st, 2024 at 5:06 PM, NIN9 Creative @.***> wrote:

@Powellga Stop posting unintelligible walls of text please. If you need to post a diff, just post a pastebin to the diff. Otherwise you could share your own fork but this is not helpful.

Agree, it'd be more helpful to see a fork or diff.

— Reply to this email directly, view it on GitHub, or unsubscribe. You are receiving this because you were mentioned.

nin9creative commented 1 month ago

not posting anymore. apologies. I just wanted to answer one person's question. After this, I do not intent to respond. sorry for the trouble. r/ Gregg … On Wednesday, July 31st, 2024 at 5:06 PM, NIN9 Creative @.***> wrote: > @Powellga Stop posting unintelligible walls of text please. If you need to post a diff, just post a pastebin to the diff. Otherwise you could share your own fork but this is not helpful. Agree, it'd be more helpful to see a fork or diff. — Reply to this email directly, view it on GitHub, or unsubscribe. You are receiving this because you were mentioned.

I mean - appreciate it, but it's hard to follow in multiple issue posts, easier thing would be as stated above.

MDKre commented 1 month ago

Unsubscribe

Max Kremzow Postfach 36 Böttgerstraße 7,13357 Berlin 0176 72678622 @.***

NIN9 Creative @.***> schrieb am Do. 1. Aug. 2024 um 14:51:

not posting anymore. apologies. I just wanted to answer one person's question. After this, I do not intent to respond. sorry for the trouble. r/ Gregg … <#m9088482101079316374> On Wednesday, July 31st, 2024 at 5:06 PM, NIN9 Creative @.***> wrote: > @Powellga https://github.com/Powellga Stop posting unintelligible walls of text please. If you need to post a diff, just post a pastebin to the diff. Otherwise you could share your own fork but this is not helpful. Agree, it'd be more helpful to see a fork or diff. — Reply to this email directly, view it on GitHub, or unsubscribe. You are receiving this because you were mentioned.

I mean - appreciate it, but it's hard to follow in multiple issue posts, easier thing would be as stated above.

— Reply to this email directly, view it on GitHub https://github.com/frdel/agent-zero/issues/12#issuecomment-2262961422, or unsubscribe https://github.com/notifications/unsubscribe-auth/AXHV4YYIHYDPBCJF6JMTGV3ZPIVL3AVCNFSM6AAAAABLURKZVOVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDENRSHE3DCNBSGI . You are receiving this because you are subscribed to this thread.Message ID: @.***>

SagaForge commented 1 week ago

Here is my attempt at the complete documentation to the best of my memory for all the changes and steps I made to rework the Dockerfile, main.py, models.py, requirements.txt, and to get Ollama and Agent Zero running within a Docker container. I attached these files to this email. Make sure you put docker file and initialize.sh in the docker folder, and models.py and requirements.txt go in the agent-zer folder. r/ gregg powell ### 1. Changes to Files #### updated Dockerfile: # Use the latest slim version of DebianFROM debian:bookworm-slim # Set ARG for platform-specific commands ARG TARGETPLATFORM # Update and install necessary packages RUN apt-get update && apt-get install -y \     python3 \     python3-pip \     python3-venv \     nodejs \     npm \     openssh-server \     sudo \     curl \     && rm -rf /var/lib/apt/lists/* # Set up SSH RUN mkdir /var/run/sshd && \     echo 'root:toor' | chpasswd && \     sed -i 's/#PermitRootLogin prohibit-password/PermitRootLogin yes/' /etc/ssh/sshd_config # Create and activate Python virtual environment ENV VIRTUAL_ENV=/opt/venv RUN python3 -m venv $VIRTUAL_ENV # Copy initial .bashrc with virtual environment activation to a temporary location COPY docker/.bashrc /etc/skel/.bashrc # Copy the script to ensure .bashrc is in the root directory COPY docker/initialize.sh /usr/local/bin/initialize.sh RUN chmod +x /usr/local/bin/initialize.sh # Ensure the virtual environment and pip setup RUN $VIRTUAL_ENV/bin/pip install --upgrade pip # Copy the application files COPY . /app # Set the working directory WORKDIR /app # Install dependencies using the virtual environment RUN $VIRTUAL_ENV/bin/pip install -r requirements.txt # Expose SSH port EXPOSE 22 # Init .bashrc CMD ["/bin/bash", "/usr/local/bin/initialize.sh"] change to models.py (includes what I have in modles.py)Ensure you have the correct API key and the model names used match those available in Ollama. import osfrom dotenv import load_dotenv from langchain_community.llms import Ollama from langchain_openai import ChatOpenAI, OpenAI, OpenAIEmbeddings from langchain_anthropic import ChatAnthropic from langchain_groq import ChatGroq from langchain_huggingface import HuggingFaceEmbeddings from langchain_google_genai import ChatGoogleGenerativeAI, HarmBlockThreshold, HarmCategory # Load environment variables load_dotenv() # Configuration DEFAULT_TEMPERATURE = 0.0 # Utility function to get API keys from environment variables def get_api_key(service):     return os.getenv(f"APIKEY{service.upper()}") or "your-api-key-here" # Factory functions for each model type def get_openai_chat(api_key=None, model="gpt-4o-mini", temperature=DEFAULT_TEMPERATURE):     api_key = api_key or get_api_key("openai")     return ChatOpenAI(model_name=model, temperature=temperature, api_key=api_key) # type: ignore def get_ollama_llama31(temperature=DEFAULT_TEMPERATURE):     return Ollama(model="llama3.1", temperature=temperature) changes to main.py (includes the entire reworked main.py)Update the initialize function to use the ollama_llama31 model. import threading, time, models, osfrom ansio import application_keypad, mouse_input, raw_input from ansio.input import InputEvent, get_input_event from agent import Agent, AgentConfig from python.helpers.print_style import PrintStyle from python.helpers.files import read_file from python.helpers import files import python.helpers.timed_input as timed_input input_lock = threading.Lock() os.chdir(files.get_abs_path("./work_dir")) #change CWD to work_dir def initialize():     # main chat model used by agents (smarter, more accurate)     chat_llm = models.get_ollama_llama31(temperature=0.2)     # utility model used for helper functions (cheaper, faster)     utility_llm = models.get_openai_chat(temperature=0)     # embedding model used for memory     embedding_llm = models.get_embedding_openai()     # agent configuration     config = AgentConfig(         chat_model = chat_llm,         utility_model = utility_llm,         embeddings_model = embedding_llm,         code_exec_docker_enabled = True,         code_exec_ssh_enabled = True,     )     # create the first agent     agent0 = Agent(number = 0, config = config)     # start the chat loop     chat(agent0) # Main conversation loop def chat(agent: Agent):     while True:         with input_lock:             timeout = agent.get_data("timeout")             if not timeout:                 PrintStyle(background_color="#6C3483", font_color="white", bold=True, padding=True).print(f"User message ('e' to leave):")                 import readline                 user_input = input("> ")                 PrintStyle(font_color="white", padding=False, log_only=True).print(f"> {user_input}")             else:                 PrintStyle(background_color="#6C3483", font_color="white", bold=True, padding=True).print(f"User message ({timeout}s timeout, 'w' to wait, 'e' to leave):")                 import readline                 user_input = timeout_input("> ", timeout=timeout)                 if not user_input:                     user_input = read_file("prompts/fw.msg_timeout.md")                     PrintStyle(font_color="white", padding=False).stream(f"{user_input}")                 else:                     user_input = user_input.strip()                     if user_input.lower() == 'w':                         user_input = input("> ").strip()                     PrintStyle(font_color="white", padding=False, log_only=True).print(f"> {user_input}")         if user_input.lower() == 'e': break         assistant_response = agent.message_loop(user_input)         PrintStyle(font_color="white", background_color="#1D8348", bold=True, padding=True).print(f"{agent.agent_name}: reponse:")         PrintStyle(font_color="white").print(f"{assistant_response}") def intervention():     if Agent.streaming_agent and not Agent.paused:         Agent.paused = True         PrintStyle(background_color="#6C3483", font_color="white", bold=True, padding=True).print(f"User intervention ('e' to leave, empty to continue):")         import readline         user_input = input("> ").strip()         PrintStyle(font_color="white", padding=False, log_only=True).print(f"> {user_input}")         if user_input.lower() == 'e': os._exit(0)         if user_input: Agent.streaming_agent.intervention_message = user_input         Agent.paused = False def capture_keys():     global input_lock     intervent = False     while True:         if intervent: intervention()         intervent = False         time.sleep(0.1)         if Agent.streaming_agent:             with input_lock, raw_input, application_keypad:                 event: InputEvent | None = get_input_event(timeout=0.1)                 if event and (event.shortcut.isalpha() or event.shortcut.isspace()):                     intervent = True                     continue def timeout_input(prompt, timeout=10):     return timed_input.timeout_input(prompt=prompt, timeout=timeout) if name == "main":     print("Initializing framework...")     threading.Thread(target=capture_keys, daemon=True).start()     initialize() Updated requirements.txt Make sure ollama is included. langchainopenai ollama ### 2. Steps to Get Ollama Running in Docker 1. Install necessary tools: (bash- in the docker container) apt-get update && apt-get install -y curl                2. Download and install Ollama: (bash - inside the docker container) curl -o ollama.tar.gz -L "https://ollama.com/download/Ollama-0.3.0-Linux.tar.gz" tar -xvzf ollama.tar.gz -C /usr/local/bin rm ollama.tar.gz            3. Verify installation:  (bash - inside the docker container) ollama --version             4. Start Ollama server: (bash - inside the docker container) ollama serve ### 3. Steps to Get Agent Zero Running in Docker with Ollama 1. Build Docker Image:  (bash - inside the docker container) docker build -t agent-zero -f docker/Dockerfile .         2. Run Docker Container:  (bash - inside the docker container) docker run -d -p 2222:22 --name agent-zero-container agent-zero           3. Access Docker Container:  (bash - inside the docker container) ssh @. -p 2222             4. Pull Ollama Model: (bash - inside the docker container) ollama pull llama3.1               5. Start Ollama Server (in another terminal): ssh @. -p 2222 ollama serve               6. Run Agent Zero: (bash - inside the docker container) /opt/venv/bin/python main.py                 On Wednesday, July 31st, 2024 at 8:33 AM, Manu Linares @.***> wrote: Can't get it to work reliably with with ollama, tried these models: - llama3-groq-tool-use:latest - llama3.1:8b - llama3:instruct - gemma2:latest - mistral-nemo:12b I'd appreciate any info if somebody got it to work reliably (models, prompts). Thanks! — Reply to this email directly, view it on GitHub, or unsubscribe. You are receiving this because you commented. ansio==0.0.1 python-dotenv==1.0.1 langchain-groq==0.1.6 langchain-huggingface==0.0.3 langchain-openai==0.1.15 langchain-community==0.2.7 langchain-anthropic==0.1.19 langchain-chroma==0.1.2 langchain-google-genai==1.0.7 webcolors==24.6.0 sentence-transformers==3.0.1 docker==7.1.0 paramiko==3.4.0 duckduckgo_search==6.1.12 inputimeout==1.0.4

@Powellga Was this fix working reliably? I've tried the steps you've given but it's still not working well, just splurts out memory injection yapper. (Ollama 0.3.8 and Llama3.1)

SupercaliG commented 1 week ago

I tried removing the prompt altogether and was able to get llama3.1 having a decent conversation. It seems like the system prompt included is too much gibberish. Try making the prompt more simplified?