search

BAAI-DCAI / Bunny

A family of lightweight multimodal models.
Apache License 2.0
866 stars 65 forks source link

Model inference speed #32

Closed Alxemade closed 4 months ago

Alxemade commented 5 months ago

hi, Great work! I tried this script huggingface-transformers, but found that the inference speed is much slower than the llava series. Do you have any relevant speed tests there?

Isaachhh commented 5 months ago

You may install flash_attn and try again.

Isaachhh commented 4 months ago

Close the issue for now if there's no further discussions. Feel free to reopen it if there's any other questions.