[Bug] [connect] 怎样才能连上我自己用vllm+fastchat部署的api呢

[erjiguan]

Search before asking

• [x] I had searched in the issues and found no similar issues.

Operating system information

Linux

Python version information

3.10

DB-GPT version

main

Related scenes

• [ ] Chat Data
• [ ] Chat Excel
• [ ] Chat DB
• [ ] Chat Knowledge
• [ ] Model Management
• [X] Dashboard
• [ ] Plugins

Installation Information

• [X] Installation From Source

• [ ] Docker Installation

• [ ] Docker Compose Installation

• [ ] Cluster Installation

• [ ] AutoDL Image

• [ ] Other

Device information

gpu

Models information

qwen-14b-chat

What happened

我用vllm+fastchat部署的模型，启动一个controller、worker和api服务，暴露了api出来，webserver要怎么才能连上呢

What you expected to happen

webserver连上vllm+fastchat本地部署的api

How to reproduce

see error

Additional context

No response

Are you willing to submit PR?

• [ ] Yes I am willing to submit a PR!

[fangyinc]

@erjiguan 你好，可以通连接 OpenAI ChatGPT 的方式去连接。 另外 DB-GPT 也支持集群和 vLLM 的部署，可以根据 Cluster Deployment试用一下。

[tianya092]

@erjiguan 老哥，搞定了吗？怎么让系统调用其他机器部署的兼容接口

[chuangzhidan]

@erjiguan 你好，可以通连接 OpenAI ChatGPT 的方式去连接。 另外 DB-GPT 也支持集群和 vLLM 的部署，可以根据 Cluster Deployment试用一下。

替换openai那个proxyllm吗？密码empty，url http://10.0.18.15:8008/v1/chat/completions这种格式？ 他好像有个baseurl .env的LLM_MODEL需要更改吗