search

itayhubara / BinaryNet.pytorch

Binarized Neural Network (BNN) for pytorch
504 stars 125 forks source link

Activation for BinaryNet #5

Open ruizhoud opened 6 years ago

ruizhoud commented 6 years ago

Hello,

I noticed that torch.nn.Hardtanh is used for activation functions in BinaryNet. This is meant to make the model trainable, as introduced in the BNN paper. However, in the inference phase (the validate() function in main_binary.py), shoudn't the activation function be changed to sign function so that the intermediate results are binary?

Thanks!

roeemz commented 6 years ago

Check out binarized_modules: https://github.com/itayhubara/BinaryNet.pytorch/blob/master/models/binarized_modules.py#L76 and https://github.com/itayhubara/BinaryNet.pytorch/blob/master/models/binarized_modules.py#L95

You will see that the input to each layer is Binarized anyway so as long as you use BinarizeConv2d or BinarizeLinear you are good.