search

QwenLM / Qwen-VL

The official repo of Qwen-VL (通义千问-VL) chat & pretrained large vision language model proposed by Alibaba Cloud.
Other
4.76k stars 359 forks source link

Will the Qwen-VL support flash attention? #235

Open jihuishan opened 8 months ago

jihuishan commented 8 months ago

起始日期 | Start Date

No response

实现PR | Implementation PR

No response

相关Issues | Reference Issues

No response

摘要 | Summary

Trying to finetune, yet running slow in training and inferring.

基本示例 | Basic Example

Just like Qwen-7B, could you support flash attention for Qwen-VL?

缺陷 | Drawbacks

It perhaps costs some effort to implement.

未解决问题 | Unresolved questions

No response

Liuziyu77 commented 3 months ago

The same Question

cxy1996 commented 1 month ago

the same question