How can I get model.layers[n].mixer.xai_b?

AmeenAli / HiddenMambaAttn

Official PyTorch Implementation of "The Hidden Attention of Mamba Models"

204 stars 12 forks source link

How can I get model.layers[n].mixer.xai_b? #12

Closed DingjieFu closed 5 months ago

DingjieFu commented 6 months ago

Hello,

Thank you for your amazing work!

I have a question regarding attention visualization, which is shown in "xai_utils.py". It appears that the Vim model dictionary doesn't have an attribute named 'xai_b'. How can I obtain it?

Thank you!