gptq reorder canceled in quip?

Cornell-RelaxML / QuIP

Code for paper: "QuIP: 2-Bit Quantization of Large Language Models With Guarantees"

339 stars 31 forks source link

gptq reorder canceled in quip? #13

Open chenzx921020 opened 6 months ago

chenzx921020 commented 6 months ago

In llama, if choose gptq method to quant, its actorder by hessian was canceled? I found some modify records in ldlq method, in function 'round_vecbal_Hsort', including Hdiag.sort. But if choose GPTQ, method is class GPTQ, not Balance, which is related to ldlq. Despite of the above, by your pre/postproc, gptq still can achieve a better result

chenzx921020 commented 5 months ago

maybe this for static group ? subsequent group quantization can realize dynamic update?