issues
search
Deelvin
/
mlc-llm
Enable everyone to develop, optimize and deploy AI models natively on everyone's devices.
https://mlc.ai/mlc-llm
Apache License 2.0
0
stars
0
forks
source link
Calibrate and use matmul error as bias
#1
Open
vvchernov
opened
8 months ago
vvchernov
commented
8 months ago
Support on calibration pipeline the measurement of error between fake-quantized matrix multiplication and original one and use it as bias in an quantized topology.
Check accuracy with/without bias on different topologies and quantization approaches.