Support using local(open-source) LLMs instead than OpenAI API as backend

melih-unsal / DemoGPT

🤖 Everything you need to create an LLM Agent—tools, prompts, frameworks, and models—all in one place.

MIT License

1.69k stars 200 forks source link

Support using local(open-source) LLMs instead than OpenAI API as backend #45

Closed wusiyaodiudiu closed 8 months ago

wusiyaodiudiu commented 1 year ago

Utilize FastChat to provide an API for the open-source LLM model, integrating it in the form of an OpenAI API interface to enhance the loading performance of the LLM model. Newly added files: demogpt/model_config.py demogpt/server/llm_api.py Modified files:" demogpt/app.py demogpt/model.py

Refs #41