search

itayhubara / BinaryNet.pytorch

Binarized Neural Network (BNN) for pytorch
505 stars 126 forks source link

add Residual in basicblock #3

Open hex0102 opened 6 years ago

hex0102 commented 6 years ago

In BasicBlock defined in resnet_binary.py, during the forward propagation, residual is cloned from the input X as Line 47 shows. And the residual is added to the results of Convs. Why need this addition? BinaryNet is supposed to work in binary form, however, the residual is in form of floating point representation. These seem to be contradicted.

roeemz commented 6 years ago

Skip connections are a major part of ResNet and therefore they are implemented here too. https://arxiv.org/abs/1512.03385

Check out binarized_modules: https://github.com/itayhubara/BinaryNet.pytorch/blob/master/models/binarized_modules.py#L76 and https://github.com/itayhubara/BinaryNet.pytorch/blob/master/models/binarized_modules.py#L95

You will see that the input to each layer is Binarized anyway so as long as you use BinarizeConv2d or BinarizeLinear you are good.