Aaronhuang-778 / BiLLM

(ICML 2024) BiLLM: Pushing the Limit of Post-Training Quantization for LLMs
https://arxiv.org/abs/2402.04291
MIT License
198 stars 14 forks source link

Question about 1-bit compression with combined binary masks #13

Open pprp opened 6 months ago

pprp commented 6 months ago

I have been studying the BiLLM algorithm described in your paper, and I have a question regarding the claimed 1-bit compression achieved through the combination of three binary masks.

In the following algorithm, you proposed the residual approximation by using two binary representation. I wonder if this method belongs to 2 bit rather than 1bit?

image