Aaronhuang-778 / BiLLM

(ICML 2024) BiLLM: Pushing the Limit of Post-Training Quantization for LLMs
https://arxiv.org/abs/2402.04291
MIT License
178 stars 13 forks source link

Question about 1-bit compression with combined binary masks #13

Open pprp opened 4 months ago

pprp commented 4 months ago

I have been studying the BiLLM algorithm described in your paper, and I have a question regarding the claimed 1-bit compression achieved through the combination of three binary masks.

In the following algorithm, you proposed the residual approximation by using two binary representation. I wonder if this method belongs to 2 bit rather than 1bit?

image