search

Aaronhuang-778 / BiLLM

(ICML 2024) BiLLM: Pushing the Limit of Post-Training Quantization for LLMs
https://arxiv.org/abs/2402.04291
MIT License
155 stars 12 forks source link

I have a question about the paper. #6

Closed shampooooo closed 4 months ago

shampooooo commented 4 months ago

Hi, Thanks for the interesting paper.

Problem: Row and Column

The size of the weight matrix W is n x m, and the per-column segmentation is performed as shown in the figure below, right? 제목 없음

The paper clearly states 'k is the number of selected columns.' Isn't m x k the correct notation? 제목 없음1

Initially, I thought this was a typo. However, seeing 'rows' continually used instead of 'columns' in later mentions and in Algorithm 1 made me wonder if I had misunderstood something.

shampooooo commented 4 months ago

Thank you for your response.

So, should the 'rows' mentioned in the algorithm all be represented as 'columns'? I have spent a lot of time trying to understand this in some way 😭.

Aaronhuang-778 commented 4 months ago

Sorry, I only noticed the second picture of your explanation, based on the equations mentioned in the paper: n represents rows and m denotes columns. So the right expression of this typo is $n \times k$.

Aaronhuang-778 commented 4 months ago

Sorry, I only noticed the second picture of your explanation, based on the equations mentioned in the paper: n represents rows and m denotes columns. So the right expression of this typo is n×k.

The details of this selection part are explained in util/autoseatch.py file, the structural selection is based on the column-wise. Your initial insight is right, and thanks again for your correction :-)