search

microsoft / LQ-Nets

LQ-Nets: Learned Quantization for Highly Accurate and Compact Deep Neural Networks
MIT License
239 stars 69 forks source link

questions about your paper and code #1

Closed brisker closed 5 years ago

brisker commented 6 years ago

in the paper you mentioned "Our goal is to find an optimal quantizer basis v ∈ RK as well as an encoding B = [b1, ..., bN] ∈ {−1, 1}K×N that minimize the quantization error". So take the activation quantization for example, did you mean that 1) each layer has its own v and its own B? 2) v is learnable (included in the computional graph) and B is not learnable but just calculated with v( both in the training and testing stage)? 3) what is the variable for? "init_thrs_multiplier" here: https://github.com/Microsoft/LQ-Nets/blob/master/learned_quantization.py#L50

chenrudan commented 6 years ago

1.Yes 2.Yes 3.The v caculates the quantization levels such as a,b(a<b). "init_thrs_multiplier" and "thrs" get the middle value of levels such as (a+b)/2. So we compare data c with this thresh (a+b)/2 we can know c will be quanted to a or b.

brisker commented 5 years ago

@chenrudan but in the code here : https://github.com/Microsoft/LQ-Nets/blob/master/learned_quantization.py#L164 every time the code enters in QuantizedWeight(), the 'basis' variable will be initailized again and again, but not using the new moving average value. Is this a bug?