search

TracecatHQ / tracecat

The open source Tines / Splunk SOAR alternative.
https://tracecat.com
GNU Affero General Public License v3.0
2.27k stars 145 forks source link

Suggestion for this task - Bring-your-own LLM (OpenAI, Mistral, Anthropic etc.) #132

Open HakaishinShwet opened 2 months ago

HakaishinShwet commented 2 months ago

There is one great open source project on GitHub called LiteLLM. https://github.com/BerriAI/litellm

If that project is integrated to this project then this task of todo check list can be easily fulfilled without struggling and read documentations for n numbers of different LLM's as LiteLLM has simple schema/structure in which people can fill info and use any LLM(100's of LLM supported by LiteLLM) easily. It created local proxy server and use open ai api structure for working so we don't need too much things and don't have to study different LLM documentation to integrate it in this solution so things will be much more easy so I would suggest you to study on it and implement it to make things much more easy for yourself and for users too :-)) Many great projects have implemented and in some i have given this same suggestion so it can be helpful for all :-)) Thankyou for this great project

topher-lo commented 2 months ago

We evaluated a bunch of options (including the one you suggested), leaning towards https://github.com/Portkey-AI/gateway

Thoughts? @HakaishinShwet

HakaishinShwet commented 2 months ago

Sorry for late response , it is also good @topher-lo but i havent tested it out personally and have just seen repo so if you are satisfied with it then it's best for this solution :-)) Thanks for sharing this one i will try myself too and there is btw litellm discord server where you can ask them questions and differences and they will be best guys to resolve any doubts or queries you may in choosing between litellm and portkey gateway