search

VRSEN / agency-swarm

The only reliable agent framework built on top of the latest OpenAI Assistants API.
https://vrsen.github.io/agency-swarm/
MIT License
2.38k stars 640 forks source link

Use local LLMs #14

Closed maxfahl closed 9 months ago

maxfahl commented 9 months ago

First of all, thank you for creating and open sourcing this awesome library. Would be awesome if we could define and use local models, maybe through the API feature of "LM Studio" or similar? The API created there is almost identical to the one of OAI.

Simplereally commented 9 months ago

Keen for this

VRSEN commented 9 months ago

This library will not support local LLMs due to several key factors:

  1. Function Calling: Essential for executing tools. OpenAI's models are specifically trained to call functions, eliminating the need for any extra prompts. This is crucial for making agents reliable in production.
  2. Document Uploads: The Assistants API enables direct document uploads and storage within each agent, which allows you to easily expand your agent swarms. Implementing this with open-source models would require a complex vector store management system.
  3. Future Capabilities: OpenAI is likely to introduce more advanced features to the Assistants API, such as built-in conversation memory, among others. With their first mover advantage, it might be extremely hard for any Open Source LLMs to ever catch up, unfortunately.
fyyyyy commented 9 months ago

What about MemGPT? https://github.com/cpacker/MemGPT