Parameter number problem of Mamba2

state-spaces / mamba

Mamba SSM architecture

Apache License 2.0

12.27k stars 1.03k forks source link

Parameter number problem of Mamba2 #435

Open zzzendurance opened 1 month ago

zzzendurance commented 1 month ago

I did not do bidirectional processing inside mamba2 (the same as Vision Mamba), I did a bidirectional work outside of the class, but the whole model has a large number of parameters, I would like to ask how to solve this, anyone has done bidirectional work inside Mamba2 for reference.

albertfgu commented 1 month ago

Yes, our official bidirectional version will be released next week, stay tuned!