Hello,
I would like to express my appreciation for the outstanding work on this project.
In the original Mamba, there is no existence of the 'z' mechanism as a gate. However, in Vim, an additional 'z' has been incorporated as a gate. Why was it designed this way? What would happen if it were removed?
Hello, I would like to express my appreciation for the outstanding work on this project.
In the original Mamba, there is no existence of the 'z' mechanism as a gate. However, in Vim, an additional 'z' has been incorporated as a gate. Why was it designed this way? What would happen if it were removed?
The code in question can be found at: https://github.com/hustvl/Vim/blob/6143d07b3dd31f904c63840a19e22d95d1124493/mamba-1p1p1/mamba_ssm/modules/mamba_simple.py#L351
Your clarification would be immensely appreciated.