search

state-spaces / mamba

Mamba SSM architecture
Apache License 2.0
12.45k stars 1.05k forks source link

Mamba2 interpretability #424

Open RedekopEP opened 2 months ago

RedekopEP commented 2 months ago

Hi! Is there an established way to get mamba interpretability, smth similar to self-attention analysis in transformers. Thank you!

tridao commented 2 months ago

You can probably construct sth similar to the attention matrix https://arxiv.org/abs/2403.01590