what‘s the self.if_bidirectional,I konw that we define bimaba_type=v1/v2 in mamba_simple.py to present we use bidirectional ssm. So what's the meaning of self.if_bidirectional in models_mamba.py,I don't know it,please help me.
thanks !
if not self.if_bidirectional:
..............
else:
# get two layers in a single for-loop
for i in range(len(self.layers) // 2): #
if self.if_rope:
hidden_states = self.rope(hidden_states)
if residual is not None and self.if_rope_residual:
residual = self.rope(residual)
hidden_states_f, residual_f = self.layers[i * 2](
hidden_states, residual, inference_params=inference_params
)
hidden_states_b, residual_b = self.layers[i * 2 + 1](
hidden_states.flip([1]), None if residual == None else residual.flip([1]), inference_params=inference_params
)
hidden_states = hidden_states_f + hidden_states_b.flip([1])
residual = residual_f + residual_b.flip([1])
what‘s the self.if_bidirectional,I konw that we define bimaba_type=v1/v2 in mamba_simple.py to present we use bidirectional ssm. So what's the meaning of self.if_bidirectional in models_mamba.py,I don't know it,please help me. thanks !