How can I use other LLM, such as LLAMA2?

wangyf456 commented 8 months ago

Do I only can use chatgpt? how can I use other LLM, such as LLAMA2. Could you give some config?

zhc7 commented 8 months ago

You may specify models in config/assignment/default.yaml. A simple example would be replacing gpt-3.5-turbo with vicuna-7b. If you wish to use other models, you need to first define them in config/agents.

另外如果想说中文也是可以的

Dhaizei commented 8 months ago

你可以使用fastchat进行加载模型（controller 和model worker），在fschat.yaml里面配置路径和使用模型的名称

zhc7 commented 7 months ago

如果仍有问题欢迎重启issue

YinSonglin1997 commented 3 weeks ago

你可以使用fastchat进行加载模型（controller 和model worker），在fschat.yaml里面配置路径和使用模型的名称

您好，我使用fastchat进行加载chatglm3-6b模型， step1 python3 -m fastchat.serve.controller step2 python3 -m fastchat.serve.model_worker --model-path /ldata/llms/chatglm3-6b step3 python3 -m fastchat.serve.openai_api_server --host 10.0.1.227 --port 30008 启动服务后，我修改了fs_agents.yaml文件，内容为

default:
  module: "src.client.agents.FastChatAgent"
  parameters:
    name: "FastChat"
    controller_address: "http://10.0.1.227:30008"
    max_new_tokens: 512
    temperature: 0

chatglm3-6b:
  parameters:
    model_name: "chatglm3-6b"

然后运行 python -m src.client.agent_test --config configs/agents/fs_agents.yaml --agent chatglm3-6b 接着新开了一个终端运行命令 python -m src.start_task -a，此时报错

mysql.connector.errors.DatabaseError: 2003 (HY000): Can't connect to MySQL server on '127.0.0.1:16018' (111)

请问您遇到过这个问题吗？还有我的步骤以及修改有问题吗？期待您的回复！

THUDM / AgentBench

How can I use other LLM, such as LLAMA2? #75