Open Kurama622 opened 5 months ago
I'm not sure if this is a bug. Maybe it’s the cumulative error caused by floating point operations, maybe there’s some difference between micro batch gradient update and mini batch.
解决了吗
解决了吗
目前我还没有得到回复,我推测是浮点数精度误差累积的问题,但是我也不完全确定
🐞 Bug
The same batch size, different micro batches, the algorithm effects are inconsistent.
I have fixed the random seed.
I set chunks equal to 2 or 4
Code that reproduces