search

shibing624 / textgen

TextGen: Implementation of Text Generation models, include LLaMA, BLOOM, GPT2, BART, T5, SongNet and so on. 文本生成模型，实现了包括LLaMA，ChatGLM，BLOOM，GPT2，Seq2Seq，BART，T5，UDA等模型的训练和预测，开箱即用。
Apache License 2.0
937 stars 109 forks source link

多卡部署 #39

Open ZTurboX opened 1 year ago

ZTurboX commented 1 year ago

想请问一下，用fastapi多卡部署生成式模型，怎样在不同卡上起服务呢

shibing624 commented 1 year ago

每个卡起一个server，再做负载均衡。

ZTurboX commented 1 year ago

每个卡起一个server，再做负载均衡。

请问有案例吗

stale[bot] commented 11 months ago

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.(由于长期不活动，机器人自动关闭此问题，如果需要欢迎提问)