Open Yisen-Feng opened 3 months ago
I also have met this problem!
The current backward pass is not deterministic (it uses atomic adds).
I think that's the pytorch cumsum, so it's out of the scope of this repo.
I think that's the pytorch cumsum, so it's out of the scope of this repo.
I see. Thank you.
Hi,
Thank you for your awesome work! I have a query regarding the reproducibility of Mamba. Even though I use
torch.use_deterministic_algorithms(True, warn_only=False)
in my code, after adding Mamba, my code lost reproducibility. I wonder if it is possible to have a deterministic Mamba.