search

Josh-XT / AGiXT

AGiXT is a dynamic AI Agent Automation Platform that seamlessly orchestrates instruction management and complex task execution across diverse AI providers. Combining adaptive memory, smart features, and a versatile plugin system, AGiXT delivers efficient and comprehensive AI solutions.
https://AGiXT.com
MIT License
2.64k stars 354 forks source link

AGIXT + Koboldcpp + GGUF models = nonsense generation #1023

Closed AliveDedSec closed 1 year ago

AliveDedSec commented 1 year ago

Description

I use koboldcpp-1.45 and different models of the GGUF format, they all generate nonsense in Agent_Interactions in any modes (chat, instruct, promt). All information is in the logs.

Visit our documentation at https://AGiXT.com 
Welcome to the AGiXT Installer!
Please choose an option:
  1. Run AGiXT (Recommended)
  2. Run AGiXT with Text Generation Web UI (NVIDIA Only)
  3. Run AGiXT with Text Generation Web UI and Stable Diffusion (NVIDIA Only)
Developer Only Options (Not recommended or supported):
  4. Run AGiXT from Main Branch
  5. Run AGiXT from Main Branch + Addons (NVIDIA Only)
  6. Run AGiXT without Docker
Manage:
  7. Disable Automatic Updates
  8. Exit

Enter your choice: 1
Running Updates...
Updating AGiXT Core...
fatal: не найден git репозиторий (или один из родительских каталогов): .git
Please wait...
Current directory: /home/agi/AGiXT
[+] Pulling 3/3
 ✔ streamlit Pulled                                                        3.3s 
 ✔ nextjs Pulled                                                           3.4s 
 ✔ agixt Pulled                                                            3.2s 
Updates Completed...
Update complete.
Starting Docker Compose...
[+] Running 3/0
 ✔ Container agixt-agixt-1      Created                                    0.0s 
 ✔ Container agixt-nextjs-1     Created                                    0.0s 
 ✔ Container agixt-streamlit-1  Created                                    0.0s 
Attaching to agixt-agixt-1, agixt-nextjs-1, agixt-streamlit-1
agixt-agixt-1      | Starting AGiXT... Please wait until you see 'Applicaton startup complete' before opening Streamlit...
agixt-agixt-1      | INFO:     Uvicorn running on http://0.0.0.0:7437 (Press CTRL+C to quit)
agixt-agixt-1      | INFO:     Started parent process [9]
agixt-nextjs-1     | 
agixt-nextjs-1     | > agixt-nextjs@0.0.35 start
agixt-nextjs-1     | > next start -p 24498
agixt-nextjs-1     | 
agixt-nextjs-1     | 
agixt-streamlit-1  | 
agixt-streamlit-1  |   You can now view your Streamlit app in your browser.
agixt-streamlit-1  | 
agixt-streamlit-1  |   Network URL: 
agixt-streamlit-1  | http://172.20.0.3:8501
agixt-streamlit-1  |   External URL: http://51.210.247.246:8501
agixt-streamlit-1  | 
agixt-streamlit-1  | 
agixt-nextjs-1     |   ▲ Next.js 13.5.3
agixt-nextjs-1     |   - Local:        http://localhost:24498
agixt-nextjs-1     | 
agixt-nextjs-1     | 
agixt-nextjs-1     |  ✓ Ready in 1233ms
agixt-agixt-1      | INFO:     Started server process [12]
agixt-agixt-1      | INFO:     Waiting for application startup.
agixt-agixt-1      | INFO:     Application startup complete.
agixt-agixt-1      | INFO:     Started server process [14]
agixt-agixt-1      | INFO:     Waiting for application startup.
agixt-agixt-1      | INFO:     Application startup complete.
agixt-agixt-1      | INFO:     Started server process [19]
agixt-agixt-1      | INFO:     Waiting for application startup.
agixt-agixt-1      | INFO:     Application startup complete.
agixt-agixt-1      | INFO:     Started server process [16]
agixt-agixt-1      | INFO:     Waiting for application startup.
agixt-agixt-1      | INFO:     Application startup complete.
agixt-agixt-1      | INFO:     Started server process [20]
agixt-agixt-1      | INFO:     Started server process [18]
agixt-agixt-1      | INFO:     Waiting for application startup.
agixt-agixt-1      | INFO:     Waiting for application startup.
agixt-agixt-1      | INFO:     Application startup complete.
agixt-agixt-1      | INFO:     Application startup complete.
agixt-agixt-1      | INFO:     Started server process [17]
agixt-agixt-1      | INFO:     Waiting for application startup.
agixt-agixt-1      | INFO:     Application startup complete.
agixt-agixt-1      | INFO:     Started server process [13]
agixt-agixt-1      | INFO:     Waiting for application startup.
agixt-agixt-1      | INFO:     Application startup complete.
agixt-agixt-1      | INFO:     Started server process [15]
agixt-agixt-1      | INFO:     Waiting for application startup.
agixt-agixt-1      | INFO:     Application startup complete.
agixt-agixt-1      | INFO:     Started server process [11]
agixt-agixt-1      | INFO:     Waiting for application startup.
agixt-agixt-1      | INFO:     Application startup complete.
agixt-agixt-1      | INFO:     172.20.0.3:50030 - "GET /api/OpenAI/conversations HTTP/1.1" 200 OK
agixt-agixt-1      | INFO:     172.20.0.3:50032 - "GET /api/conversation HTTP/1.1" 200 OK
agixt-agixt-1      | INFO:     172.20.0.3:50048 - "GET /api/OpenAI/conversations HTTP/1.1" 200 OK
agixt-agixt-1      | INFO:     172.20.0.3:50052 - "GET /api/conversation HTTP/1.1" 200 OK
agixt-agixt-1      | INFO:     172.20.0.3:50064 - "GET /api/agent HTTP/1.1" 200 OK
agixt-agixt-1      | INFO:     172.20.0.3:50072 - "GET /api/OpenAI/conversations HTTP/1.1" 200 OK
agixt-agixt-1      | INFO:     172.20.0.3:50084 - "GET /api/conversation HTTP/1.1" 200 OK
agixt-agixt-1      | INFO:     172.20.0.3:50094 - "GET /api/agent HTTP/1.1" 200 OK
agixt-agixt-1      | INFO:     172.20.0.3:50106 - "GET /api/agent HTTP/1.1" 200 OK
agixt-agixt-1      | INFO:     172.20.0.3:50114 - "GET /api/provider HTTP/1.1" 200 OK
agixt-agixt-1      | INFO:     172.20.0.3:50120 - "GET /api/embedding_providers HTTP/1.1" 200 OK
agixt-agixt-1      | INFO:     127.0.0.1:50070 - "GET /api/chain HTTP/1.1" 200 OK
agixt-agixt-1      | INFO:     172.20.0.3:50130 - "GET /api/extensions/settings HTTP/1.1" 200 OK
agixt-agixt-1      | INFO:     172.20.0.3:42934 - "GET /api/agent HTTP/1.1" 200 OK
agixt-agixt-1      | INFO:     172.20.0.3:49140 - "GET /api/agent HTTP/1.1" 200 OK
agixt-agixt-1      | INFO:     172.20.0.3:49156 - "GET /api/agent HTTP/1.1" 200 OK
agixt-agixt-1      | INFO:     172.20.0.3:49168 - "GET /api/agent/AIM HTTP/1.1" 200 OK
agixt-agixt-1      | INFO:     172.20.0.3:49174 - "GET /api/agent HTTP/1.1" 200 OK
agixt-agixt-1      | INFO:     127.0.0.1:45932 - "GET /api/chain HTTP/1.1" 200 OK
agixt-agixt-1      | INFO:     172.20.0.3:49176 - "GET /api/agent/AIM HTTP/1.1" 200 OK
agixt-agixt-1      | 2023-10-01 19:20:23,883 | INFO | Getting options for provider: kobold
agixt-agixt-1      | INFO:     172.20.0.3:49186 - "GET /api/provider/kobold HTTP/1.1" 200 OK
agixt-agixt-1      | INFO:     127.0.0.1:45940 - "GET /api/chain HTTP/1.1" 200 OK
agixt-agixt-1      | INFO:     172.20.0.3:49190 - "GET /api/extensions HTTP/1.1" 200 OK
agixt-agixt-1      | INFO:     172.20.0.3:49202 - "GET /api/agent/AIM/command HTTP/1.1" 200 OK
agixt-agixt-1      | INFO:     172.20.0.3:41256 - "GET /api/AIM/conversations HTTP/1.1" 200 OK
agixt-agixt-1      | INFO:     172.20.0.3:41264 - "GET /api/conversation HTTP/1.1" 200 OK
agixt-agixt-1      | INFO:     172.20.0.3:41276 - "GET /api/agent HTTP/1.1" 200 OK
agixt-agixt-1      | INFO:     172.20.0.3:48222 - "GET /api/AIM/conversations HTTP/1.1" 200 OK
agixt-agixt-1      | INFO:     172.20.0.3:48236 - "GET /api/conversation HTTP/1.1" 200 OK
agixt-agixt-1      | INFO:     172.20.0.3:48242 - "GET /api/agent HTTP/1.1" 200 OK
agixt-agixt-1      | INFO:     172.20.0.3:48244 - "GET /api/AIM/conversations HTTP/1.1" 200 OK
agixt-agixt-1      | INFO:     172.20.0.3:48248 - "GET /api/conversation HTTP/1.1" 200 OK
agixt-agixt-1      | INFO:     172.20.0.3:48264 - "GET /api/agent HTTP/1.1" 200 OK
agixt-agixt-1      | INFO:     127.0.0.1:36040 - "GET /api/chain HTTP/1.1" 200 OK
agixt-agixt-1      | INFO:     127.0.0.1:36060 - "GET /api/chain HTTP/1.1" 200 OK
agixt-agixt-1      | INFO:     127.0.0.1:36052 - "GET /api/agent/AIM HTTP/1.1" 200 OK
agixt-agixt-1      | 2023-10-01 19:22:24,186 | INFO | CONTEXT RESULTS: 5
agixt-agixt-1      | 2023-10-01 19:22:25,517 | WARNING | Embedding collection is empty.
agixt-agixt-1      | 2023-10-01 19:22:26,137 | WARNING | Embedding collection is empty.
agixt-agixt-1      | 2023-10-01 19:22:26,617 | WARNING | Embedding collection is empty.
agixt-agixt-1      | 2023-10-01 19:22:32,250 | INFO | FORMATTED PROMPT:   
agixt-agixt-1      | You are AIM, an expert at answering the users question.  Use available context help answer the users questions.
agixt-agixt-1      | 
agixt-agixt-1      |   
agixt-agixt-1      | 
agixt-agixt-1      | Recent conversation history for context: 
agixt-agixt-1      |  August 14, 2023 10:56 PM USER: What are the most interesting things you can tell me about AGiXT? 
agixt-agixt-1      |  August 14, 2023 10:56 PM OpenAI: AGiXT is an artificial intelligence automation platform that combines adaptive memory handling, smart features, and a versatile plugin system to optimize AI performance and enhance task completion. It offers several key features, including:
agixt-agixt-1      | 
agixt-agixt-1      | 1. Context and Token Management: AGiXT optimizes AI performance by efficiently managing long-term and short-term memory.
agixt-agixt-1      | 
agixt-agixt-1      | 2. Smart Instruct: This advanced feature enables AI to comprehend, plan, and execute tasks effectively, integrating web search, planning strategies, and instruction execution.
agixt-agixt-1      | 
agixt-agixt-1      | 3. Interactive Chat & Smart Chat: AGiXT provides a user-friendly chat interface for dynamic conversational tasks. The Smart Chat feature integrates AI with web research to deliver accurate and relevant responses.
agixt-agixt-1      | 
agixt-agixt-1      | 4. Task Execution & Smart Task Management: AGiXT efficiently manages and executes complex tasks by breaking them down into sub-tasks and using AI-driven agents for optimization and avoiding redundancy.
agixt-agixt-1      | 
agixt-agixt-1      | 5. Chain Management: AGiXT handles chains or series of linked commands, enabling the automation of complex workflows and processes.
agixt-agixt-1      | 
agixt-agixt-1      | 6. Web Browsing & Command Execution: AGiXT offers advanced capabilities to browse the web and execute commands, providing a more interactive AI experience.
agixt-agixt-1      | 
agixt-agixt-1      | 7. Multi-Provider Compatibility: AGiXT seamlessly integrates with leading AI providers such as OpenAI GPT series, Hugging Face Huggingchat, GPT4All, GPT4Free, Oobabooga Text Generation Web UI, Kobold, Llamacpp, FastChat, Google Bard, Bing, and more.
agixt-agixt-1      | 
agixt-agixt-1      | 8. Versatile Plugin System & Code Evaluation: AGiXT supports various AI models and robust code evaluation, making it useful for programming tasks.
agixt-agixt-1      | 
agixt-agixt-1      | 9. Docker Deployment: AGiXT simplifies setup and maintenance through Docker deployment.
agixt-agixt-1      | 
agixt-agixt-1      | 10. Audio-to-Text & Text-to-Speech Options: AGiXT integrates with Hugging Face for audio-to-text transcription and offers multiple text-to-speech choices.
agixt-agixt-1      | 
agixt-agixt-1      | 11. Platform Interoperability & AI Agent Management: AGiXT allows easy interaction with popular platforms like Twitter, GitHub, Google, DALL-E, and more. It also provides granular control over agent abilities and supports custom prompts.
agixt-agixt-1      | 
agixt-agixt-1      | 12. RESTful API: AGiXT offers a FastAPI-powered RESTful API for seamless integration with external applications and services.
agixt-agixt-1      | 
agixt-agixt-1      | 13. Expanding AI Support: AGiXT continually updates to include new AI providers and services, ensuring it stays at the forefront of AI technology.
agixt-agixt-1      | 
agixt-agixt-1      | For more details and documentation, you can visit the AGiXT GitHub repository and explore the different AGiXT components and capabilities. 
agixt-agixt-1      |  October 01, 2023 02:01 PM USER: What are the most interesting things you can tell me about AGiXT? 
agixt-agixt-1      |  October 01, 2023 02:03 PM USER: What are the most interesting things you can tell me about AGiXT? 
agixt-agixt-1      |   
agixt-agixt-1      | 
agixt-agixt-1      | User's latest input: What are the most interesting things you can tell me about AGiXT? 
agixt-agixt-1      | 
agixt-agixt-1      | 
agixt-agixt-1      | 2023-10-01 19:22:46,585 | INFO | Response:   Let q = -4 + 3.95. Sort -1/6, 2, q in descending order.
agixt-agixt-1      | 
agixt-agixt-1      | Answer: 2, q, -1/6
agixt-agixt-1      | 
agixt-agixt-1      | INFO:     172.20.0.3:48270 - "POST /api/agent/AIM/prompt HTTP/1.1" 200 OK
agixt-agixt-1      | INFO:     172.20.0.3:39598 - "GET /api/AIM/conversations HTTP/1.1" 200 OK
agixt-agixt-1      | INFO:     172.20.0.3:39602 - "GET /api/conversation HTTP/1.1" 200 OK
agixt-agixt-1      | INFO:     172.20.0.3:39618 - "GET /api/agent HTTP/1.1" 200 OK
agixt-agixt-1      | INFO:     172.20.0.3:44260 - "GET /api/AIM/conversations HTTP/1.1" 200 OK
agixt-agixt-1      | INFO:     172.20.0.3:44268 - "GET /api/conversation HTTP/1.1" 200 OK
agixt-agixt-1      | INFO:     172.20.0.3:44272 - "GET /api/agent HTTP/1.1" 200 OK
agixt-agixt-1      | INFO:     172.20.0.3:44284 - "GET /api/prompt/categories HTTP/1.1" 200 OK
agixt-agixt-1      | INFO:     172.20.0.3:44296 - "GET /api/prompt/Default HTTP/1.1" 200 OK
agixt-agixt-1      | INFO:     172.20.0.3:44310 - "GET /api/prompt/Default/Custom%20Input HTTP/1.1" 200 OK
agixt-agixt-1      | INFO:     172.20.0.3:44322 - "GET /api/prompt/Default/Custom%20Input/args HTTP/1.1" 200 OK
agixt-agixt-1      | INFO:     172.20.0.3:44336 - "GET /api/AIM/conversations HTTP/1.1" 200 OK
agixt-agixt-1      | INFO:     172.20.0.3:44340 - "GET /api/conversation HTTP/1.1" 200 OK
agixt-agixt-1      | INFO:     172.20.0.3:44344 - "GET /api/agent HTTP/1.1" 200 OK
agixt-agixt-1      | INFO:     172.20.0.3:44346 - "GET /api/prompt/categories HTTP/1.1" 200 OK
agixt-agixt-1      | INFO:     172.20.0.3:44362 - "GET /api/prompt/Default HTTP/1.1" 200 OK
agixt-agixt-1      | INFO:     172.20.0.3:44378 - "GET /api/prompt/Default/Custom%20Input HTTP/1.1" 200 OK
agixt-agixt-1      | INFO:     172.20.0.3:44394 - "GET /api/prompt/Default/Custom%20Input/args HTTP/1.1" 200 OK
agixt-agixt-1      | INFO:     172.20.0.3:38388 - "GET /api/AIM/conversations HTTP/1.1" 200 OK
agixt-agixt-1      | INFO:     172.20.0.3:38394 - "GET /api/conversation HTTP/1.1" 200 OK
agixt-agixt-1      | INFO:     172.20.0.3:38400 - "GET /api/agent HTTP/1.1" 200 OK
agixt-agixt-1      | INFO:     172.20.0.3:38412 - "GET /api/prompt/categories HTTP/1.1" 200 OK
agixt-agixt-1      | INFO:     172.20.0.3:38424 - "GET /api/prompt/Default HTTP/1.1" 200 OK
agixt-agixt-1      | INFO:     172.20.0.3:38428 - "GET /api/prompt/Default/Task%20Execution HTTP/1.1" 200 OK
agixt-agixt-1      | INFO:     172.20.0.3:38440 - "GET /api/prompt/Default/Task%20Execution/args HTTP/1.1" 200 OK
agixt-agixt-1      | INFO:     172.20.0.3:38444 - "GET /api/AIM/conversations HTTP/1.1" 200 OK
agixt-agixt-1      | INFO:     172.20.0.3:38460 - "GET /api/conversation HTTP/1.1" 200 OK
agixt-agixt-1      | INFO:     172.20.0.3:38462 - "GET /api/agent HTTP/1.1" 200 OK
agixt-agixt-1      | INFO:     172.20.0.3:38472 - "GET /api/prompt/categories HTTP/1.1" 200 OK
agixt-agixt-1      | INFO:     172.20.0.3:38484 - "GET /api/prompt/Default HTTP/1.1" 200 OK
agixt-agixt-1      | INFO:     172.20.0.3:38496 - "GET /api/prompt/Default/Task%20Execution HTTP/1.1" 200 OK
agixt-agixt-1      | INFO:     172.20.0.3:38508 - "GET /api/prompt/Default/Task%20Execution/args HTTP/1.1" 200 OK
agixt-agixt-1      | INFO:     172.20.0.3:38510 - "GET /api/AIM/conversations HTTP/1.1" 200 OK
agixt-agixt-1      | INFO:     172.20.0.3:38526 - "GET /api/conversation HTTP/1.1" 200 OK
agixt-agixt-1      | INFO:     172.20.0.3:38536 - "GET /api/agent HTTP/1.1" 200 OK
agixt-agixt-1      | INFO:     172.20.0.3:38538 - "GET /api/prompt/categories HTTP/1.1" 200 OK
agixt-agixt-1      | INFO:     172.20.0.3:38554 - "GET /api/prompt/Default HTTP/1.1" 200 OK
agixt-agixt-1      | INFO:     172.20.0.3:38556 - "GET /api/prompt/Default/Task%20Execution HTTP/1.1" 200 OK
agixt-agixt-1      | INFO:     172.20.0.3:38564 - "GET /api/prompt/Default/Task%20Execution/args HTTP/1.1" 200 OK
agixt-agixt-1      | INFO:     127.0.0.1:43264 - "GET /api/chain HTTP/1.1" 200 OK
agixt-agixt-1      | INFO:     127.0.0.1:41138 - "GET /api/agent/AIM HTTP/1.1" 200 OK
agixt-agixt-1      | 2023-10-01 19:24:01,562 | INFO | CONTEXT RESULTS: 5
agixt-agixt-1      | 2023-10-01 19:24:02,135 | INFO | FORMATTED PROMPT: 
agixt-agixt-1      | **To execute a command, use the example below, it will be replaced with the commands output for the user. You can execute a command anywhere in your response and the commands will be executed in the order you use them.**
agixt-agixt-1      | #execute_command("Name of Command", {"arg1": "val1", "arg2": "val2"})
agixt-agixt-1      | What are the most interesting things you can tell me about AGiXT?
agixt-agixt-1      | 
agixt-agixt-1      | You have the following constraints:
agixt-agixt-1      | 1. If you are unsure how you previously did something or want to recall past events, thinking about similar events will help you remember.
agixt-agixt-1      | 2. No user assistance.
agixt-agixt-1      | 3. Use any available context to help make decisions.
agixt-agixt-1      | 4. Only use available commands.
agixt-agixt-1      | 5. The working directory is ./WORKSPACE , all accessible files and folders are in this directory or a subdirectory.
agixt-agixt-1      | 
agixt-agixt-1      | RESPOND IN THE FOLLOWING JSON FORMAT ONLY! If there are no commands, simply make the commands section an empty object like {}.
agixt-agixt-1      | ```JSON
agixt-agixt-1      | {
agixt-agixt-1      |     "response": "Your response to the task.",
agixt-agixt-1      |     "commands": {
agixt-agixt-1      |         "command_name": {
agixt-agixt-1      |             "arg1": "val1",
agixt-agixt-1      |             "arg2": "val2"
agixt-agixt-1      |         },
agixt-agixt-1      |         "command_name2": {
agixt-agixt-1      |             "arg1": "val1",
agixt-agixt-1      |             "arg2": "val2",
agixt-agixt-1      |             "argN": "valN"
agixt-agixt-1      |         }
agixt-agixt-1      |     }
agixt-agixt-1      | }
agixt-agixt-1      | ```
agixt-agixt-1      | 
agixt-agixt-1      | 2023-10-01 19:35:09,518 | INFO | Response: 0000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000010000000000010000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000
agixt-agixt-1      | INFO:     172.20.0.3:38574 - "POST /api/agent/AIM/prompt HTTP/1.1" 200 OK
agixt-agixt-1      | INFO:     172.20.0.3:41806 - "GET /api/AIM/conversations HTTP/1.1" 200 OK
agixt-agixt-1      | INFO:     172.20.0.3:41822 - "GET /api/conversation HTTP/1.1" 200 OK
agixt-agixt-1      | INFO:     172.20.0.3:41826 - "GET /api/agent HTTP/1.1" 200 OK
agixt-agixt-1      | INFO:     172.20.0.3:41834 - "GET /api/prompt/categories HTTP/1.1" 200 OK
agixt-agixt-1      | INFO:     172.20.0.3:41846 - "GET /api/prompt/Default HTTP/1.1" 200 OK
agixt-agixt-1      | INFO:     172.20.0.3:41858 - "GET /api/prompt/Default/Task%20Execution HTTP/1.1" 200 OK
agixt-agixt-1      | INFO:     172.20.0.3:41866 - "GET /api/prompt/Default/Task%20Execution/args HTTP/1.1" 200 OK
agixt-agixt-1      | INFO:     172.20.0.3:50460 - "GET /api/AIM/conversations HTTP/1.1" 200 OK
agixt-agixt-1      | INFO:     172.20.0.3:50464 - "GET /api/conversation HTTP/1.1" 200 OK
agixt-agixt-1      | INFO:     172.20.0.3:50466 - "GET /api/agent HTTP/1.1" 200 OK
agixt-agixt-1      | INFO:     172.20.0.3:50480 - "GET /api/prompt/categories HTTP/1.1" 200 OK
agixt-agixt-1      | INFO:     172.20.0.3:50490 - "GET /api/prompt/Default HTTP/1.1" 200 OK
agixt-agixt-1      | INFO:     172.20.0.3:50494 - "GET /api/prompt/Default/Task%20Execution HTTP/1.1" 200 OK
agixt-agixt-1      | INFO:     172.20.0.3:50500 - "GET /api/prompt/Default/Task%20Execution/args HTTP/1.1" 200 OK
agixt-agixt-1      | INFO:     172.20.0.3:49764 - "GET /api/AIM/conversations HTTP/1.1" 200 OK
agixt-agixt-1      | INFO:     172.20.0.3:49766 - "GET /api/conversation HTTP/1.1" 200 OK
agixt-agixt-1      | INFO:     172.20.0.3:49780 - "GET /api/agent HTTP/1.1" 200 OK
agixt-agixt-1      | INFO:     172.20.0.3:40266 - "GET /api/AIM/conversations HTTP/1.1" 200 OK
agixt-agixt-1      | INFO:     172.20.0.3:40278 - "GET /api/conversation HTTP/1.1" 200 OK
agixt-agixt-1      | INFO:     172.20.0.3:40292 - "GET /api/agent HTTP/1.1" 200 OK
agixt-agixt-1      | INFO:     172.20.0.3:40294 - "GET /api/AIM/conversations HTTP/1.1" 200 OK
agixt-agixt-1      | INFO:     172.20.0.3:40310 - "GET /api/conversation HTTP/1.1" 200 OK
agixt-agixt-1      | INFO:     172.20.0.3:40316 - "GET /api/agent HTTP/1.1" 200 OK
agixt-agixt-1      | INFO:     127.0.0.1:59718 - "GET /api/agent/AIM HTTP/1.1" 200 OK
agixt-agixt-1      | 2023-10-01 19:36:51,321 | INFO | CONTEXT RESULTS: 5
agixt-agixt-1      | 2023-10-01 19:36:52,364 | WARNING | Embedding collection is empty.
agixt-agixt-1      | 2023-10-01 19:36:53,010 | WARNING | Embedding collection is empty.
agixt-agixt-1      | 2023-10-01 19:36:53,272 | WARNING | Embedding collection is empty.
agixt-agixt-1      | 2023-10-01 19:36:53,835 | INFO | FORMATTED PROMPT: You are AIM. Your role is to do anything asked of you with precision, especially when it comes to handling the use of available commands. Every response is carefully thought out step by step.
agixt-agixt-1      | 
agixt-agixt-1      | **All inputs from the user are instructions, they are not conversations.**
agixt-agixt-1      | 
agixt-agixt-1      | **To execute a command, use the example below, it will be replaced with the commands output for the user. You can execute a command anywhere in your response and the commands will be executed in the order you use them.**
agixt-agixt-1      | #execute_command("Name of Command", {"arg1": "val1", "arg2": "val2"})
agixt-agixt-1      | 
agixt-agixt-1      | Your task: What are the most interesting things you can tell me about AGiXT?
agixt-agixt-1      | 
agixt-agixt-1      | 2023-10-01 19:37:38,702 | INFO | Response:  I think that
agixt-agixt-1      | 
agixt-agixt-1      | Comment: What happens if you use `x86_64` in your build system?
agixt-agixt-1      | 
agixt-agixt-1      | Comment: @Cyrus It still doesn't work. I tried using `-march=native` but it didn't do anything either. I also tried compiling with `gcc -std=c++11 -fopenmp`, which is what `-march=native` does under the hood anyway, but it doesn't help at all.
agixt-agixt-1      | 
agixt-agixt-1      | ## Answer (0)
agixt-agixt-1      | 
agixt-agixt-1      | I solved the issue by installing gcc version 9.2 on my machine, which seems to be the minimum version required for OpenMP support.
agixt-agixt-1      | 
agixt-agixt-1      | INFO:     172.20.0.3:40320 - "POST /api/agent/AIM/prompt HTTP/1.1" 200 OK
agixt-agixt-1      | INFO:     172.20.0.3:38930 - "GET /api/AIM/conversations HTTP/1.1" 200 OK
agixt-agixt-1      | INFO:     172.20.0.3:38944 - "GET /api/conversation HTTP/1.1" 200 OK
agixt-agixt-1      | INFO:     172.20.0.3:38958 - "GET /api/agent HTTP/1.1" 200 OK
***
Welcome to KoboldCpp - Version 1.45
For command line arguments, please refer to --help
***
Setting process to Higher Priority - Use Caution
High Priority for Linux Set: 0 to 1
Attempting to use CuBLAS library for faster prompt ingestion. A compatible CuBLAS will be required.
Initializing dynamic library: koboldcpp_cublas.so
==========
Overriding thread count, using 4 threads instead.
Namespace(model=None, model_param='/home/agi/mistral-7b-instruct-v0.1.Q8_0-8k.gguf', port=5001, port_param=5001, host='0.0.0.0', launch=False, lora=None, config=None, threads=4, blasthreads=4, psutil_set_threads=True, highpriority=True, contextsize=32768, blasbatchsize=32, ropeconfig=[0.0, 10000.0], stream=True, smartcontext=False, unbantokens=True, bantokens=None, usemirostat=None, forceversion=0, nommap=False, usemlock=True, noavx2=False, debugmode=False, skiplauncher=False, hordeconfig=None, noblas=False, useclblast=None, usecublas=['lowvram'], gpulayers=12, tensor_split=None, onready='', multiuser=False, foreground=False)
==========
Loading model: /home/agi/mistral-7b-instruct-v0.1.Q8_0-8k.gguf 
[Threads: 4, BlasThreads: 4, SmartContext: False]

---
Identified as LLAMA model: (ver 6)
Attempting to Load...
---
Using automatic RoPE scaling (scale:1.000, base:10000.0)
System Info: AVX = 1 | AVX2 = 0 | AVX512 = 0 | AVX512_VBMI = 0 | AVX512_VNNI = 0 | FMA = 0 | NEON = 0 | ARM_FMA = 0 | F16C = 1 | FP16_VA = 0 | WASM_SIMD = 0 | BLAS = 1 | SSE3 = 1 | SSSE3 = 1 | VSX = 0 | 
ggml_init_cublas: found 1 CUDA devices:
  Device 0: NVIDIA GeForce GTX 1050 Ti, compute capability 6.1
llama_model_loader: loaded meta data with 20 key-value pairs and 291 tensors from /home/agi/mistral-7b-instruct-v0.1.Q8_0-8k.gguf (version GGUF V2 (latest))
llm_load_print_meta: format           = GGUF V2 (latest)
llm_load_print_meta: arch             = llama
llm_load_print_meta: vocab type       = SPM
llm_load_print_meta: n_vocab          = 32000
llm_load_print_meta: n_merges         = 0
llm_load_print_meta: n_ctx_train      = 32768
llm_load_print_meta: n_embd           = 4096
llm_load_print_meta: n_head           = 32
llm_load_print_meta: n_head_kv        = 8
llm_load_print_meta: n_layer          = 32
llm_load_print_meta: n_rot            = 128
llm_load_print_meta: n_gqa            = 4
llm_load_print_meta: f_norm_eps       = 0.0e+00
llm_load_print_meta: f_norm_rms_eps   = 1.0e-05
llm_load_print_meta: n_ff             = 14336
llm_load_print_meta: freq_base_train  = 10000.0
llm_load_print_meta: freq_scale_train = 1
llm_load_print_meta: model type       = 7B
llm_load_print_meta: model ftype      = unknown, may not work
llm_load_print_meta: model params     = 7.24 B
llm_load_print_meta: model size       = 7.17 GiB (8.50 BPW) 
llm_load_print_meta: general.name   = mistralai_mistral-7b-instruct-v0.1
llm_load_print_meta: BOS token = 1 '<s>'
llm_load_print_meta: EOS token = 2 '</s>'
llm_load_print_meta: UNK token = 0 '<unk>'
llm_load_print_meta: LF token  = 13 '<0x0A>'
llm_load_tensors: ggml ctx size =    0.09 MB
llm_load_tensors: using CUDA for GPU acceleration
llm_load_tensors: mem required  = 4686.36 MB
llm_load_tensors: offloading 12 repeating layers to GPU
llm_load_tensors: offloaded 12/35 layers to GPU
llm_load_tensors: VRAM used: 2652.38 MB
warning: failed to mlock 139264000-byte buffer (after previously locking 0 bytes): Cannot allocate memory
Try increasing RLIMIT_MLOCK ('ulimit -l' as root).
...................................................................................................
llama_new_context_with_model: n_ctx      = 32768
llama_new_context_with_model: freq_base  = 10000.0
llama_new_context_with_model: freq_scale = 1
llama_new_context_with_model: kv self size  = 4096.00 MB
llama_new_context_with_model: compute buffer total size = 139.38 MB
llama_new_context_with_model: VRAM scratch buffer: 133.50 MB
llama_new_context_with_model: total VRAM used: 2785.88 MB (model: 2652.38 MB, context: 133.50 MB)
Load Model OK: True
Embedded Kobold Lite loaded.
Starting Kobold HTTP Server on port 5001
Please connect to custom endpoint at http://0.0.0.0:5001

Input: {"prompt": "  \nYou are AIM, an expert at answering the users question.  Use available context help answer the users questions.\n\n  \n\nRecent conversation history for context: \n August 14, 2023 10:56 PM USER: What are the most interesting things you can tell me about AGiXT? \n August 14, 2023 10:56 PM OpenAI: AGiXT is an artificial intelligence automation platform that combines adaptive memory handling, smart features, and a versatile plugin system to optimize AI performance and enhance task completion. It offers several key features, including:\n\n1. Context and Token Management: AGiXT optimizes AI performance by efficiently managing long-term and short-term memory.\n\n2. Smart Instruct: This advanced feature enables AI to comprehend, plan, and execute tasks effectively, integrating web search, planning strategies, and instruction execution.\n\n3. Interactive Chat & Smart Chat: AGiXT provides a user-friendly chat interface for dynamic conversational tasks. The Smart Chat feature integrates AI with web research to deliver accurate and relevant responses.\n\n4. Task Execution & Smart Task Management: AGiXT efficiently manages and executes complex tasks by breaking them down into sub-tasks and using AI-driven agents for optimization and avoiding redundancy.\n\n5. Chain Management: AGiXT handles chains or series of linked commands, enabling the automation of complex workflows and processes.\n\n6. Web Browsing & Command Execution: AGiXT offers advanced capabilities to browse the web and execute commands, providing a more interactive AI experience.\n\n7. Multi-Provider Compatibility: AGiXT seamlessly integrates with leading AI providers such as OpenAI GPT series, Hugging Face Huggingchat, GPT4All, GPT4Free, Oobabooga Text Generation Web UI, Kobold, Llamacpp, FastChat, Google Bard, Bing, and more.\n\n8. Versatile Plugin System & Code Evaluation: AGiXT supports various AI models and robust code evaluation, making it useful for programming tasks.\n\n9. Docker Deployment: AGiXT simplifies setup and maintenance through Docker deployment.\n\n10. Audio-to-Text & Text-to-Speech Options: AGiXT integrates with Hugging Face for audio-to-text transcription and offers multiple text-to-speech choices.\n\n11. Platform Interoperability & AI Agent Management: AGiXT allows easy interaction with popular platforms like Twitter, GitHub, Google, DALL-E, and more. It also provides granular control over agent abilities and supports custom prompts.\n\n12. RESTful API: AGiXT offers a FastAPI-powered RESTful API for seamless integration with external applications and services.\n\n13. Expanding AI Support: AGiXT continually updates to include new AI providers and services, ensuring it stays at the forefront of AI technology.\n\nFor more details and documentation, you can visit the AGiXT GitHub repository and explore the different AGiXT components and capabilities. \n October 01, 2023 02:01 PM USER: What are the most interesting things you can tell me about AGiXT? \n October 01, 2023 02:03 PM USER: What are the most interesting things you can tell me about AGiXT? \n  \n\nUser's latest input: What are the most interesting things you can tell me about AGiXT? \n", "max_context_length": 2048, "max_length": 2048, "temperature": 0.7}

Processing Prompt (1 / 1 tokens)
Generating (43 / 2047 tokens)
(EOS token triggered!)
Time Taken - Processing:0.4s (390ms/T), Generation:13.9s (323ms/T), Total:14.3s (3.0T/s)
Output:   Let q = -4 + 3.95. Sort -1/6, 2, q in descending order.

Answer: 2, q, -1/6

Input: {"prompt": "\n**To execute a command, use the example below, it will be replaced with the commands output for the user. You can execute a command anywhere in your response and the commands will be executed in the order you use them.**\n#execute_command(\"Name of Command\", {\"arg1\": \"val1\", \"arg2\": \"val2\"})\nWhat are the most interesting things you can tell me about AGiXT?\n\nYou have the following constraints:\n1. If you are unsure how you previously did something or want to recall past events, thinking about similar events will help you remember.\n2. No user assistance.\n3. Use any available context to help make decisions.\n4. Only use available commands.\n5. The working directory is ./WORKSPACE , all accessible files and folders are in this directory or a subdirectory.\n\nRESPOND IN THE FOLLOWING JSON FORMAT ONLY! If there are no commands, simply make the commands section an empty object like {}.\n```JSON\n{\n    \"response\": \"Your response to the task.\",\n    \"commands\": {\n        \"command_name\": {\n            \"arg1\": \"val1\",\n            \"arg2\": \"val2\"\n        },\n        \"command_name2\": {\n            \"arg1\": \"val1\",\n            \"arg2\": \"val2\",\n            \"argN\": \"valN\"\n        }\n    }\n}\n```", "max_context_length": 2048, "max_length": 2048, "temperature": 0.7}

Processing Prompt (1 / 1 tokens)
Generating (2047 / 2047 tokens)
Time Taken - Processing:0.3s (321ms/T), Generation:667.0s (326ms/T), Total:667.3s (3.1T/s)
Output: 0000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000010000000000010000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000

Input: {"prompt": "You are AIM. Your role is to do anything asked of you with precision, especially when it comes to handling the use of available commands. Every response is carefully thought out step by step.\n\n**All inputs from the user are instructions, they are not conversations.**\n\n**To execute a command, use the example below, it will be replaced with the commands output for the user. You can execute a command anywhere in your response and the commands will be executed in the order you use them.**\n#execute_command(\"Name of Command\", {\"arg1\": \"val1\", \"arg2\": \"val2\"})\n\nYour task: What are the most interesting things you can tell me about AGiXT?", "max_context_length": 2048, "max_length": 2048, "temperature": 0.7}

Processing Prompt (1 / 1 tokens)
Generating (146 / 2047 tokens)
(EOS token triggered!)
Time Taken - Processing:1.9s (1929ms/T), Generation:42.9s (294ms/T), Total:44.8s (3.3T/s)
Output:  I think that

Comment: What happens if you use `x86_64` in your build system?

Comment: @Cyrus It still doesn't work. I tried using `-march=native` but it didn't do anything either. I also tried compiling with `gcc -std=c++11 -fopenmp`, which is what `-march=native` does under the hood anyway, but it doesn't help at all.

## Answer (0)

I solved the issue by installing gcc version 9.2 on my machine, which seems to be the minimum version required for OpenMP support.

Steps to Reproduce the Bug

I use Select Provider kobold Select Helper Agent gpt4free Any models generate responses in Agent_Interactions that do not correspond to requests, in any modes, chat, hint, instruction. Please check it out for yourself!

Expected Behavior

Adequate feedback from AGIXT + Koboldcpp + GGUF models

Operating System

Python Version

Environment Type - Connection

Runtime environment

Acknowledgements

Josh-XT commented 1 year ago

Is the model generating any less nonsense when you're interacting directly with Kobold without AGiXT?

AliveDedSec commented 1 year ago

Is the model generating any less nonsense when you're interacting directly with Kobold without AGiXT?

Josh-XT local models with AGiXT + koboldcpp do not work fine for me. Generate off-topic responses in AGiXT. Something is wrong, I suspect your instructions are in the wrong format. Maybe I'm wrong. But in the koboldcpp and SillyTavern-1.10.4 web interfaces the models work perfectly.

https://github.com/Josh-XT/AGiXT/issues/907#issuecomment-1740273002

Josh-XT commented 1 year ago

Hello @pegas07770 ! I think I understand the issue now. AGiXT did not have prompt prefix/suffix set up for the kobold provider. I just added this support in #1042 . This would certainly cause the LLM to misbehave because it is unsure whos turn it is to talk or what is happening, it isn't aware that it is a conversation without the prefix/suffix, it just thinks it needs to complete the text it was prompted with.

Alternatively, I've recently added a Local provider in the new web UI that will allow you to choose a model from a drop down to generate text on, then it will download the model if it isn't already downloaded and generate text with it. It uses llamacpp like Kobold does on the back end and works pretty well. The local provider will grab the prompt template for the model and apply it automatically when it downloads the model. More about the local provider in the Local-LLM repository if you're interested.

Hopefully that helps!