HolyBayes / pytorch_ard

Pytorch implementation of Variational Dropout Sparsifies Deep Neural Networks
MIT License
83 stars 16 forks source link

Cannot create Conv2dARD with bias=True #4

Closed Animatory closed 4 years ago

Animatory commented 4 years ago

Due to Conv2dARD creates bias before initialization of the parent class (torch.nn.Conv2d)

if bias: bias = Parameter(torch.Tensor(out_channels))
else: bias = None  
super(Conv2dARD, self).__init__(in_channels, out_channels, kernel_size, stride,  
           padding, dilation, groups, bias)

torch.Tensor goes into In super().init therefore these lines in torch/nn/modules/conv.py are failed

42 if bias:
43    self.bias = Parameter(torch.Tensor(out_channels))

with RuntimeError: bool value of Tensor with more than one value is ambiguous

Python 3.7.4 PyTorch 1.2

N2606 commented 4 years ago

Hi @Animatory , same problem here. Have you fixed it?

HolyBayes commented 4 years ago

Fixed! Conv2dARD.bias removed - it immediately crashes convergence. Use batchnorm instead