NVIDIA / cutlass

CUDA Templates for Linear Algebra Subroutines
Other
5.72k stars 982 forks source link

[BUG] Wrong assertion in integer_subbyte.h #1949

Open Algy opened 1 week ago

Algy commented 1 week ago

Describe the bug I got an Assertion failure while I tried 55_hopper_int4_fp8_gemm

This was due to the following incorrect assertion logic at [this line]:(https://github.com/NVIDIA/cutlass/blob/8aa95dbb888be6d81c6fbf7169718c5244b53227/include/cutlass/integer_subbyte.h#L96):

      [[maybe_unused]] constexpr int lower_bound = -(1 << (Bits - 1));
      [[maybe_unused]] constexpr int upper_bound = (1 << (Bits - 1)) - 1;
      assert(value >= lower_bound);
      assert(value < upper_bound); # HERE

The assertion should be assert(value <= upper_bound) as the upper_bound is inclusive.

foreverlms commented 6 days ago

I got the same error and this assertion should be what you said. I think it should be: https://github.com/NVIDIA/cutlass/blob/8aa95dbb888be6d81c6fbf7169718c5244b53227/include/cutlass/integer_subbyte.h#L95-L96 :

  assert(value > lower_bound);
  assert(value <= upper_bound); 
Algy commented 6 days ago

@foreverlms the lower bound is inclusive as well since the domain of an integer is [-2**(Bits-1), 2**(Bits-1) - 1]. For example the domain of 8-bit integer (i.e., Bits=8) is [-128, 127].