Problem on understanding the Code and corresponding paper

[zxzhijia]

I read the paper "Binarized Neural Networks". It's a very nice paper. But it's hard for me to find the g_{ak}<--g{ak^b}\cdot 1{|a_k|<=1} part (Algorithm 1 in the paper) in this code. I supposed this is the code of that paper. Could anyone help me on this?

[MatthieuCourbariaux]

But it's hard for me to find the g_{ak}<--g{ak^b}\cdot 1{|a_k|<=1} part (Algorithm 1 in the paper) in this code

Here is the part of the code you are interested in: https://github.com/MatthieuCourbariaux/BinaryNet/blob/master/Train-time/binary_net.py#L37

Keep in mind we are using Theano's automatic differentiation.

[zxzhijia]

@MatthieuCourbariaux Thank you very much for your answer. But I don't know which part do the 1_{|a_k|<=1}. Shouldn't there be a if function, which compare the {|a_k|} with 1? Or is that exist in Theano's automatic differentiation?

[MatthieuCourbariaux]

But I don't know which part do the 1_{|a_k|<=1}. Shouldn't there be a if function, which compare the {|a_k|} with 1? Or is that exist in Theano's automatic differentiation?

1_{|a_k|<=1} is the gradient of the clip function. It is taken cared of by Theano's automatic differentiation.

[zxzhijia]

@MatthieuCourbariaux I see. Thank you very much for your explanation. It helps a lot!