search

X-PLUG / mPLUG-Owl

mPLUG-Owl: The Powerful Multi-modal Large Language Model Family
https://www.modelscope.cn/studios/damo/mPLUG-Owl
MIT License
2.25k stars 171 forks source link

Slow Inference #134

Open srivivtcs opened 1 year ago

srivivtcs commented 1 year ago

Hi, the inference with the current code is very slow. Takes more than 30 mins for a single inference on a v100 gpu. How could we speedup the inference?

Zhoues commented 1 year ago

125

This issue I created may be useful to you 😃