search

hpcaitech / EnergonAI

Large-scale model inference.
Apache License 2.0
630 stars 90 forks source link

does EnergonAI support gpt model with int8 quantitation in model parallel? #158

Open dearowen opened 2 years ago

dearowen commented 2 years ago

does EnergonAI support gpt model with int8 quantitation in model parallel?

minghaoBD commented 2 years ago

Same question. And under other parallel paradigms like tensor/pipeline parallelism.