issues
search
microsoft
/
TransformerCompression
For releasing code related to compression methods for transformers, accompanying our publications
MIT License
354
stars
31
forks
source link
QuaRot bugfixes
#156
Closed
nailimixaM
closed
3 months ago
nailimixaM
commented
3 months ago
GPU OOM fix: flash attention forward pass should not
actually
do KV caching, we emulate it by quantizing and immediately dequantizing the Ks and Vs.
Various PPL mismatches: missing
symmetric
arg so always True by default caused some issues, and similar bugs
Added tests and lots of refactoring to GPTQ.
symmetric
arg so always True by default caused some issues, and similar bugsAdded tests and lots of refactoring to GPTQ.