search

microsoft / autogen

A programming framework for agentic AI 🤖
https://microsoft.github.io/autogen/
Creative Commons Attribution 4.0 International
30.05k stars 4.38k forks source link

Local Open source LLMs talk to "Uber GPT-4 LLM" when stuck #382

Closed stevenbaert closed 1 day ago

stevenbaert commented 10 months ago

I love the idea of having AutoGen creating a team of Open Source Local LLMs with a project manager, scripter, etc on my Mac. Since the agent LLMs are local, they can chat for hours and come up with a solution without any issue.

However, there is a concern that the quality of local LLMs might be lower than that of GPT-4 or Claude. So, why not dedicate an expert like GPT-4 to assist when local LLMs encounter challenges? This Uber GPT-4 LLM could serve as a project manager, overseeing things when it matters the most.

This combination could be the ideal mix to elevate a local AutoGen to the next level. Additionally, integrating MemGPT would enhance context understanding, making it less likely for the system to seek confirmations or insights from the user, as it would already be well-acquainted with the user's past projects and working methods, thanks to MemGPT.

gagb commented 10 months ago

Great idea @stevenbaert! Perhaps you can detect poor quality via code or LLMs and then use function calls to call the uber model for feedback. This notebook here is very relevant to your use case at least for the first part: https://github.com/microsoft/autogen/blob/main/notebook/agentchat_two_users.ipynb

sonichi commented 10 months ago

@JieyuZ2 has done some exploration of this idea, before MemGPT is available.

JieyuZ2 commented 10 months ago

In EcoAssistant (https://github.com/JieyuZ2/EcoAssistant/), we explore the idea of using GPT-4 as back-up for weaker models and leverage memory to help future tasks

stevenbaert commented 9 months ago

Love it, thanks! Will experiment with it and give any feedback if I see of added value for you.