search

OpenBMB / XAgent

An Autonomous LLM Agent for Complex Task Solving
https://blog.x-agent.net/blog/xagent/
Apache License 2.0
8.06k stars 827 forks source link

gguf models please #319

Open silasdao opened 10 months ago

silasdao commented 10 months ago

Feature Description / 功能描述

GGUF models please for XAgentLLM / 请描述您希望添加或改进的功能。

Goal / 目标

Inclusion of more users on Mac OS :D. / 请说明您希望通过这个功能实现的目标。

Expected Behavior / 预期行为

GGUF is the preferred format for metal inferencing. / 请描述您期望的功能行为。

Alternative Solutions / 其他解决方案

Ask TheBloke? / 如果您已经考虑了其他解决方案，请在此处提供。

Additional Notes / 其他备注

I'd really love to try the XAgentLLM soooooooo badly. / 如果有其他补充信息，请在此处添加。

Umpire2018 commented 9 months ago

Noted.

kratosok commented 7 months ago

A way support gguf models might also be through allowing OPENAI_BASE_URL to be overridden in Xagent for the openai client:

https://blog.devgenius.io/an-engineering-trick-to-use-local-llms-60a1b973225d

This would allow people to use Xagent with another app serving up GGUF or other models as a openai endpoint (llama.cpp,localai,openrouter,etc).