search

facebookresearch / kill-the-bits

Code for: "And the bit goes down: Revisiting the quantization of neural networks"
Other
636 stars 124 forks source link

How does dynamic X determine fixed Q(W)? #37

Closed bigdan12 closed 3 years ago

bigdan12 commented 3 years ago

dear author: I'm studying your papers.I have one question for your work. image as we all know.X is not fixed. Different X might make different Q(W) How you deal it?

pierrestock commented 3 years ago

Hi sfbby,

Thanks for your interest in our work and sorry for the late reply. Indeed, you are right, different values of x gives different values of q(w_j). To deal with it, we simply sample among a very large batch of input activations x at each EM step, see here.

Hope this helps,

Pierre