This is minor, but it causes nan values in my experiments.
As far as I understand, the CLIPMIN bound in quantizer.py is introduced to prevent the scale from being too small and causing division by zero. However, after the scale is clamped in the right range, it is immediately set back to the original value. In the current implementation, it makes the most sense to remove line 145 and just set CLIPMIN to 0 if that is desired in any given case.
This is minor, but it causes
nan
values in my experiments.As far as I understand, the
CLIPMIN
bound inquantizer.py
is introduced to prevent the scale from being too small and causing division by zero. However, after the scale is clamped in the right range, it is immediately set back to the original value. In the current implementation, it makes the most sense to remove line 145 and just setCLIPMIN
to 0 if that is desired in any given case.31 might also be related.