I'm on a Volta GPU, I'm aware that flash attention is not compatible with them but I've seen that the kosmos 2.5 requirements.txt contains "xformers", but I haven't seen any implementation in the kosmos 2.5 code. Do you plan to use xformers as a fallback when flash_attn is not installed?
I'm on a Volta GPU, I'm aware that flash attention is not compatible with them but I've seen that the kosmos 2.5 requirements.txt contains "xformers", but I haven't seen any implementation in the kosmos 2.5 code. Do you plan to use xformers as a fallback when flash_attn is not installed?