issues
search
wejoncy
/
QLLM
A general 2-8 bits quantization toolbox with GPTQ/AWQ/HQQ, and export to onnx/onnx-runtime easily.
Apache License 2.0
150
stars
15
forks
source link
improve .cpu() with non_blocking
#95
Closed
wejoncy
closed
9 months ago