Thanks for the great work and releasing the codebase! I have one question regarding the Mamba module initialization.
From the code, I see its linear layers are initialized here and here, then the weights and biases are overwritten here and here, which zeros out the mamba linear layers' biases. So this line actually can be deleted? Please share your insights if it's intended. Thanks!
Dear authors,
Thanks for the great work and releasing the codebase! I have one question regarding the Mamba module initialization.
From the code, I see its linear layers are initialized here and here, then the weights and biases are overwritten here and here, which zeros out the mamba linear layers' biases. So this line actually can be deleted? Please share your insights if it's intended. Thanks!