search

jbpayton / llm-auto-forge

A langchain based tool to allow agents to dynamically create, use, store, and retrieve tools to solve real world problems
MIT License
119 stars 9 forks source link

Local LLMs #1

Closed veryVANYA closed 10 months ago

veryVANYA commented 10 months ago

Any chance of integrating local LLMs through GPT4ALL, Oogabooga, LocalAI or connect to different model APIs (Claude)?

jbpayton commented 10 months ago

Thanks and stay tuned (I am AFK for a couple of days) The lowest lying fruit for me would be allowing the user to specify models for the lower level elements. This would allow everything compatible with the Langchain chat API to interoperable.

However your mileage may very as I will not likely have the capability to test them all, as right now, the only local model that I know of that is capable of “agentic behavior” is llama 70b. In terms of online hosted models Claude 2 should be good.

jbpayton commented 10 months ago

I removed the typing hints for OpenAI llms. While the examples still use them, other models could be used, provided that they implmented chat specific elements. While this doesnt directly address OogaBooga, you should be able to make use of langchain's GPT4All, and other model integrations (see list here ) - of course, your mileage may vary as to how much success you will have, due to the lack of emergent agentic behavior in some of the simpler models.

jbpayton commented 10 months ago

Closed, however, if this continues to trip people up, I may need to migrate to something other than langchain in the medium/long term.

jmanhype commented 10 months ago

flowise

Closed, however, if this continues to trip people up, I may need to migrate to something other than langchain in the medium/long term.

flowise