search

AmeenAli / HiddenMambaAttn

Official PyTorch Implementation of "The Hidden Attention of Mamba Models"
174 stars 9 forks source link

cross attention #13

Open zhending111 opened 2 weeks ago

zhending111 commented 2 weeks ago

Hello, could you tell me how Mamba performs cross-attention operations on images within a batch? I'm not sure if you've researched this area.

Itamarzimm commented 1 week ago

Hi @zhending111, please refer to item 2 in https://github.com/AmeenAli/HiddenMambaAttn/issues/7#issuecomment-2138932441 and https://github.com/state-spaces/mamba/issues/229

zhending111 commented 1 week ago

Thanks!!!