Support more LLMs - Githubissues

rnadigital / agentcloud

Agent Cloud is like having your own GPT builder with a bunch extra goodies. The GUI features 1) RAG pipeline which can natively embed 260+ datasources 2) Create Conversational apps (like GPTs) 3) Create Multi Agent process automation apps (crewai) 4) Tools 5) Teams+user permissions. Get started fast with Docker and our install.sh

https://agentcloud.dev

GNU Affero General Public License v3.0

421 stars 90 forks source link

Support more LLMs #142

Open chymian opened 2 months ago

chymian commented 2 months ago

Is your feature request related to a problem? Please describe. Local LLMs are not suficent supported.

Describe the solution you'd like since most of the local loader, like ollama, ooba's, etc all support openAI API >= 1.0, we would just need a BASE_URL field to fill in our local path to API. the LLMstudio (which is not opensource.) entry had such a field, but it's gone.

Describe alternatives you've considered Build in litellm.ai as access to 100+ llms

Additional context

anada10 commented 2 months ago

Thanks for raising @chymian this is a high priority ticket at the moment. We'll keep you posted.

tomlynchRNA commented 1 month ago

llama2 on ollama, works but its a little bit dumb (cant pick the right tool). working on improving this and adding support for many more.

image

![image](https://github.com/rnadigital/agentcloud/assets/111718755/08c97c46-609e-47b3-abb8-1779d934d33f) ![image](https://github.com/rnadigital/agentcloud/assets/111718755/6de145d2-67ee-4699-bf6e-84edad033d07) ![image](https://github.com/rnadigital/agentcloud/assets/111718755/9045b15a-074e-4a37-86c7-d9f4a68849c6)

tomlynchRNA commented 1 month ago

@charl3sj priority list for models to implement:

[x] vertex https://python.langchain.com/docs/integrations/chat/google_vertex_ai_palm/
[x] groq https://python.langchain.com/docs/integrations/chat/groq/
[x] anthropic https://python.langchain.com/docs/integrations/chat/anthropic/
[ ] huggingface https://python.langchain.com/docs/integrations/chat/huggingface/
[x] cohere https://python.langchain.com/docs/integrations/chat/cohere/
[ ] mistral https://python.langchain.com/docs/integrations/chat/mistralai/

chymian commented 1 month ago

@charl3sj priority list for models to implement:

[ ] vertex https://python.langchain.com/docs/integrations/chat/google_vertex_ai_palm/

[x] groq https://python.langchain.com/docs/integrations/chat/groq/

[x] anthropic https://python.langchain.com/docs/integrations/chat/anthropic/

[ ] huggingface https://python.langchain.com/docs/integrations/chat/huggingface/

[x] cohere https://python.langchain.com/docs/integrations/chat/cohere/

[ ] mistral https://python.langchain.com/docs/integrations/chat/mistralai/

did you have a look at litellm.ai? one solution to rule them all ;)