Could toochine be a skill broker?

[solvaholic]

There's a pattern in https://github.com/yoheinakajima/babyagi/tree/main/classic/babyfoxagi I haven't yet grokked. To me it looks like a skills orchestrator. I think it's also described as blending rules-based and generative: https://towardsdatascience.com/generative-ai-design-patterns-a-comprehensive-guide-41425a40d7d0#19ee

Could that be what toochine does? If babyagi does similar, and it's a pattern in a comprehensive guide, then I think other solutions are available. If that is what toochine does, then why use toochine instead of those other solutions?

Simplifying the front end of that diagram a bit, the pattern I have in mind looks like this:

flowchart TD
    A[Rule Creation]
    B(User Query)
    C{Rules Based<br \>Engine}
    D[Generative AI]
    E{Validation}
    F[Log & Feedback]
    G(Final Output)
    A -->|Pre-populated Rules| C
    B --> C
    C --> D
    C --> E
    D --> E
    E -->|Invalid| F
    F -->|Adjustments| C
    E -->|Valid| G
    G --> F

[solvaholic]

Oh that's interesting. Tweaking the labels a bit I think technical support could be implemented similarly. It stands out to me that 1) the special sauce is between the user and the orchestrator and 2) the Support Engineer doesn't have to do the validation.

flowchart TD
    A[Policy and Procedures]
    B(User Query)
    C{Knowledgebase,<br \>Process, Tools}
    D[Support<br \>Engineer]
    E{Validation}
    F[Log & Feedback]
    G(Solution)
    A -->|Automations<br \>And Rules| C
    B -->|Special<br \>Sauce| C
    C --> D
    C --> E
    D --> E
    E -->|Invalid| F
    F -.->|Adjustments| C
    E -->|Valid| G
    G --> F
    C -.->|Adjustments| A

The special sauce I think of is, like, imagining the context. Maybe it's implemented as the first rule?:

• Understand the user's message N different ways
• Ask for clarification until it's clear what's intended
• Phrase the user's message as a problem
• What must be true, for this problem to occur?
• What is the expected behavior?
• What is the actual behavior?

[solvaholic]

It stands out to me that 1) the special sauce is between the user and the orchestrator and 2) the Support Engineer doesn't have to do the validation.

This doc describes using OpenAI with your data. The potential usage pattern it mentions reminded me of this thread: https://learn.microsoft.com/en-us/azure/ai-services/openai/concepts/use-your-data?tabs=ai-search#potential-usage-patterns

You can define a role that you want your assistant. For example, if you are building a support bot, you can add "You are an expert incident support assistant that helps users solve new issues.".

[solvaholic]

Examples of using AutoGen to define multi-agent system: https://www.deeplearning.ai/short-courses/ai-agentic-design-patterns-with-autogen/ https://microsoft.github.io/autogen/docs/Examples/

Babyfoxagi iterated on self-improvement. From the X thread:

[solvaholic]

Additional vocabulary, research, and prior art, summarized from the PDFs by Microsoft Copilot in Edge browser:

Navigating Complexity: Orchestrated Problem Solving with Multi-Agent LLMs

Here are the key points from the paper:

• Problem Decomposition: The paper introduces a method where an orchestrating LLM breaks down complex problems into smaller, manageable sub-problems, which are then solved by specialized LLM agents.

• Multi-Agent Systems: The approach leverages multiple LLM agents, each specialized in different tasks, to collaboratively solve the decomposed sub-problems.

• Orchestration LLM: This central LLM interacts with users to understand the problem, decomposes it, assigns sub-problems to specialized agents, and compiles their solutions into a comprehensive answer.

• Scalability and Efficiency: The framework aims to enhance the problem-solving capabilities of LLMs by addressing token limitations and improving scalability and efficiency in handling complex and vague problems.

Self-Reflection in LLM Agents: Effects on Problem-Solving Performance

Here are some key areas to focus on based on the paper:

• Self-Reflection: Study how self-reflection can improve problem-solving performance in LLMs. Understand different types of self-reflection like Retry, Keywords, Advice, Explanation, Instructions, Solution, and Composite.

• Chain of Thought (CoT): Learn about CoT prompting and how it helps LLMs in reasoning and problem-solving tasks.

• Feedback Mechanisms: Explore various feedback types (internal vs. external, scalar vs. natural language) and strategies (model fine-tuning, output re-ranking, self-correction) to enhance LLM performance.

• LLM Models: Familiarize yourself with popular LLMs like GPT-4, Llama 2 70B, and Gemini 1.5 Pro, and their unique strengths and weaknesses.

LLM Harmony: Multi-Agent Communication for Problem Solving

Here are the key points from the paper "LLM Harmony: Multi-Agent Communication for Problem Solving":

• Multi-Agent Communication Framework: The paper introduces a framework where multiple LLM agents, each with a distinct persona, collaborate to solve problems. This approach is inspired by the CAMEL model.

• Chain-of-Thought Prompting: This technique is used to guide the agents in breaking down complex problems into manageable steps, enhancing their problem-solving capabilities.

• Experimentation and Results: The framework was tested on arithmetic reasoning and commonsense reasoning tasks, showing improved performance over traditional single-agent models.

• Applications and Limitations: The framework's versatility allows it to be applied to various problem-solving contexts, though it still faces challenges like context limits and the need for diverse training data.