search

AmeenAli / HiddenMambaAttn

Official PyTorch Implementation of "The Hidden Attention of Mamba Models"
204 stars 12 forks source link

cross attention #13

Closed zhending111 closed 4 months ago

zhending111 commented 5 months ago

Hello, could you tell me how Mamba performs cross-attention operations on images within a batch? I'm not sure if you've researched this area.

Itamarzimm commented 4 months ago

Hi @zhending111, please refer to item 2 in https://github.com/AmeenAli/HiddenMambaAttn/issues/7#issuecomment-2138932441 and https://github.com/state-spaces/mamba/issues/229

zhending111 commented 4 months ago

Thanks!!!