search

Holmeswww / AgentKit

An intuitive LLM prompting framework for multifunctional agents, by explicitly constructing a complex "thought process" from simple natural language prompts.
Creative Commons Attribution 4.0 International
280 stars 29 forks source link

local models #20

Closed westlongtime closed 5 months ago

westlongtime commented 5 months ago

how to bulid AgentKit without using api? I want to load my local LLM

Holmeswww commented 5 months ago

LLM_API_FUNCTION can be any LLM API function that takes msg:list and shrink_idx:int, and outputs llm_result:str and usage:dict. Where msg is a prompt (OpenAI format by default), and shrink_idx:int is an index at which the LLM should reduce the length of the prompt in case of overflow.

You can write your custom LLM function by extending https://github.com/Holmeswww/AgentKit/blob/main/src/agentkit/llm_api/base.py