Open oreo0906 opened 1 month ago
we did not investigate group quantization in this codebase, which is why it is not supported. but the method is compatible with group quantization. but incoherence and grouping aim to do similar things. Incoherence reduces overall variation in the weights via a random orthogonal matmul, while grouping reduces the effect of variation in the weights when quantizing by adding additional scaling params.
I confused the method ldlq_Rg dont support group quantization.