search

THUDM / CodeGeeX4

CodeGeeX4-ALL-9B, a versatile model for all AI software development scenarios, including code completion, code interpreter, web search, function calling, repository-level Q&A and much more.
https://codegeex.cn
Apache License 2.0
1.53k stars 118 forks source link

一机多gpu卡启动codegeex4失败 #60

Open xiaoyezi320 opened 3 months ago

xiaoyezi320 commented 3 months ago

服务器有3张T4卡，每张T4卡显存16G，codegeex4本地化部署启动至少需要19G左右到显存，启动时却只使用一张卡，报了oom异常，请问下如何才能在启动时指定使用多张卡启动模型？

hyh19962008 commented 1 month ago

https://github.com/THUDM/ChatGLM-6B里有个utils.py，可以参考一下