[DOC]: What is the hardware used in th Energon-AI inference acceleration comparison ?

hpcaitech / ColossalAI

Making large AI models cheaper, faster and more accessible

https://www.colossalai.org

Apache License 2.0

38.78k stars 4.34k forks source link

[DOC]: What is the hardware used in th Energon-AI inference acceleration comparison ? #3665

Open lz02k opened 1 year ago

lz02k commented 1 year ago

📚 The doc issue

In the Inference (Energon-AI) Demo, what is the hardware used in th Energon-AI inference acceleration ? Can you show the gpu memory?

JThh commented 1 year ago

Quoting answers from Slack: A100 and you may take a look at bloom (176b) model for memory usage (complete GPT3 training requires more memory than what our test environment can afford).