search

crazyyanchao / llmcompiler

LLMCompiler is an Agent Architecture designed to speed up the execution of agent tasks by executing them quickly in the DAG. It also saves the cost of redundant token use by reducing the number of calls to the LLM.
Apache License 2.0
18 stars 3 forks source link

Calling open-source LLMs in the planning and joining step #5

Open nooralahzadeh opened 3 months ago

nooralahzadeh commented 3 months ago

Is there any way to call open-source LLMs in the graph specially in planning and joining steps?

crazyyanchao commented 3 months ago

@nooralahzadeh Refer to the custom LLM code under llmcompiler\custom_llms to implement the LLM calling method you need, and then pass it to RunLLMCompiler.

curiothinker commented 1 month ago

I tried using a number of models other than OpenAI, but most of them did not work. One went as far and failed at joiner.inovke with " AttributeError: 'NoneType' object has no attribute 'thought' ". Any advice?