How to inference BLOOM-176B by multi-node multi-card？

hpcaitech / EnergonAI

Large-scale model inference.

Apache License 2.0

630 stars 90 forks source link

How to inference BLOOM-176B by multi-node multi-card？ #189

Open vicwer opened 1 year ago

vicwer commented 1 year ago

HI，Thank you very much for your work, I can now do bloom-560m inference on single node 4 cards by example, how to use multi-node multi-card (like 4x4x32GB v100GPU) inference bloom-176b？

sharlec commented 1 year ago

same asking here

chenbo90 commented 12 months ago

me too! how to use multi-node multi-card inference? Does Energonai support ?