search

Josh-XT / AGiXT

AGiXT is a dynamic AI Agent Automation Platform that seamlessly orchestrates instruction management and complex task execution across diverse AI providers. Combining adaptive memory, smart features, and a versatile plugin system, AGiXT delivers efficient and comprehensive AI solutions.
https://AGiXT.com
MIT License
2.64k stars 354 forks source link

Adding Petals as provider #857

Closed UndefinedConstant closed 1 year ago

UndefinedConstant commented 1 year ago

Feature/Improvement Description

Petals is a project for collaboratively running llm's with high parameter counts on consumer hardware and works similar to torrents and crypto mining pools where every user of the swarm hosts and processes a small part of the language model. It is a way of sharing ones gpu and cpu with the swarm in return for high inference speeds on big models when doing requests ourselves, and ofc the ability to run them "locally" in the first place

currently supported models LLaMA-2 70b and 70B-Chat, LLaMA-65B, Guanaco-65B and BLOOM-176B

https://github.com/bigscience-workshop/petals https://github.com/petals-infra

Proposed Solution

add petals as ai provider

Acknowledgements

Josh-XT commented 1 year ago

https://github.com/AGiXT/hub/blob/main/providers/petals.py

Added, thanks!