SelectiveScanEasy forward result have Nan

MzeroMiko / mamba-mini

An efficient pytorch implementation of selective scan in one file, works with both cpu and gpu, with corresponding mathematical derivation. It is probably the code which is the most close to selective_scan_cuda in mamba.

64 stars 1 forks source link

SelectiveScanEasy forward result have Nan #6

Open regainOWO opened 3 months ago

regainOWO commented 3 months ago

this project is amazing! i change the SelectiveScanOflex to SelectiveScanEasy for cpu inference, but the result have Nan, i found it cause by tmp_dtBus_div_rAts = (dtBus / rAts), the rAts have zero value, how to fix this?

MzeroMiko commented 3 months ago

It is only a toy version of code. For code that is appliable in practise, use mamba-2 instead.