Add GPTQ - Githubissues

microsoft / TransformerCompression

For releasing code related to compression methods for transformers, accompanying our publications

MIT License

354 stars 31 forks source link

Add GPTQ #157

Closed nailimixaM closed 3 months ago

nailimixaM commented 3 months ago

Experiments:

E2E (symmetric weights, asymmetric acts and KV)

A4W4KV4 GPTQ QuaRot ppl: 6.04 vs 6.10 in paper
A6W6KV6 GPTQ QuaRot ppl: 5.52 vs 5.52 in paper
A8W8KV8 GPTQ QuaRot ppl: 5.50 vs 5.50 in paper

Weight only, asymmetric

W4 RTN ppl: 7.02 vs 6.99
W4 GPTQ ppl: 9.62 vs 8.25 (both of these anomalous, should be at least better than RTN)
W4 RTN QuaRot ppl: 6.87 vs 6.76
W4 GPTQ QuaRot ppl: 5.61 vs 5.60

Weight only, symmetric

W4 RTN ppl: 6.99
W4 GPTQ ppl: NaN (??)
W4 RTN QuaRot ppl: 6.85
W4 GPTQ QuaRot ppl: 5.61