search

VRSEN / agency-swarm

The only reliable agent framework built on top of the latest OpenAI Assistants API.
https://vrsen.github.io/agency-swarm/
MIT License
2.72k stars 722 forks source link

Feature Request for Agency Swarm: Advanced Local AI Integration with LiteLLM, LLAVA, and Ollama #29

Closed sfdrada closed 10 months ago

sfdrada commented 10 months ago

Feature Request for Agency Swarm: Advanced Local AI Integration

Overview

Enhance Agency Swarm with LiteLLM, LLAVA, and potential Ollama integration for private, cost-effective local AI processing.

Details

  1. LiteLLM Integration:
    • Local NLP support with OpenAI API compatibility.
  2. LLAVA Integration:
    • Specialized local image processing via LiteLLM.
  3. Cost Efficiency:
    • Reduced operational expenses compared to cloud-based services.
  4. Future Ollama Integration:
    • Multiple local LLM support with easy model switching.
  5. Community and Documentation:
    • Comprehensive guides and community-driven expansions.
  6. Compatibility and Testing:
    • Smooth integration and optimal performance assurance.

Impact

Significant functionality boost for local AI processing, enhancing Agency Swarm's versatility and cost-effectiveness.

matte0fabr0 commented 10 months ago

Don't believe this is possible because using assistants API unfortunately..

VRSEN commented 10 months ago

For a few key reasons, there are no plans to support local LLMS:

https://github.com/VRSEN/agency-swarm/issues/14#issuecomment-1837780000