XLarge模型部署需要多大显存

PaddlePaddle / Knover

Large-scale open domain KNOwledge grounded conVERsation system based on PaddlePaddle

Apache License 2.0

673 stars 131 forks source link

Closed guijuzhejiang closed 1 year ago

guijuzhejiang commented 1 year ago

感谢分享的XLarge模型，请问部署一个11B参数的XLarge模型至少需要大多显存？

sserdoubleh commented 1 year ago

部署fp32的模型需要两张 V100 32G，如果使用 fp16可以做到单张 V100 32G部署如果想使用高速的推理可以参考这里： https://github.com/PaddlePaddle/PaddleNLP/tree/develop/model_zoo/plato-xl

guijuzhejiang commented 1 year ago

谢谢，关闭问题