LLMServer Generate Error, Please CheckErrorInfo.: Expected all tensors to be on the same device, but found at least two devices, cuda:2 and cuda:1!

eosphoros-ai / DB-GPT

AI Native Data App Development framework with AWEL(Agentic Workflow Expression Language) and Agents

http://docs.dbgpt.cn

MIT License

13.06k stars 1.71k forks source link

Open caowenhero opened 1 year ago

caowenhero commented 1 year ago

运行报错，目前是不支持多卡运行吗？

fangyinc commented 1 year ago

运行报错，目前是不支持多卡运行吗？

支持多卡的，辛苦提供一下启动的相关的信息（启动方式、gpu配置以及什么模型等）。

xiaoma1990-git commented 1 month ago

是用docker compose启动的 gpu是NVIDIA GeForce RTX 2080 模型:glm-4-9b-chat text2vec-large-chinese vicuna-13b-v1.5