Closed zxzhijia closed 7 years ago
But it's hard for me to find the g_{ak}<--g{ak^b}\cdot 1{|a_k|<=1} part (Algorithm 1 in the paper) in this code
Here is the part of the code you are interested in: https://github.com/MatthieuCourbariaux/BinaryNet/blob/master/Train-time/binary_net.py#L37
Keep in mind we are using Theano's automatic differentiation.
@MatthieuCourbariaux Thank you very much for your answer. But I don't know which part do the 1_{|a_k|<=1}. Shouldn't there be a if function, which compare the {|a_k|} with 1? Or is that exist in Theano's automatic differentiation?
But I don't know which part do the 1_{|a_k|<=1}. Shouldn't there be a if function, which compare the {|a_k|} with 1? Or is that exist in Theano's automatic differentiation?
1_{|a_k|<=1} is the gradient of the clip function. It is taken cared of by Theano's automatic differentiation.
@MatthieuCourbariaux I see. Thank you very much for your explanation. It helps a lot!
I read the paper "Binarized Neural Networks". It's a very nice paper. But it's hard for me to find the g_{ak}<--g{ak^b}\cdot 1{|a_k|<=1} part (Algorithm 1 in the paper) in this code. I supposed this is the code of that paper. Could anyone help me on this?