Deelvin / mlc-llm

Enable everyone to develop, optimize and deploy AI models natively on everyone's devices.
https://mlc.ai/mlc-llm
Apache License 2.0
0 stars 0 forks source link

Calibrate and use matmul error as bias #1

Open vvchernov opened 8 months ago

vvchernov commented 8 months ago
  1. Support on calibration pipeline the measurement of error between fake-quantized matrix multiplication and original one and use it as bias in an quantized topology.
  2. Check accuracy with/without bias on different topologies and quantization approaches.