Should the weights be masked at forward() ?

[bangawayoo]

Hi, thank you for sharing your implementation. I have noticed that the weights are masked out at every forwarding step such as in line 146 in models/resnet.py: self.conv1.weight.data = torch.mul(self.conv1.weight, self.mask1.weight)

From my understanding of the paper, it seems the weights are not masked out to prevent prematurely pruning weights, which is also reflected in their algorithm in Appendix A.

I see that your implementation closely reproduces their result. Would you kindly give me some intuition on why the weights should be masked at every forward step?

Thanks in advance.

[INCHEON-CHO]

Thanks for your reply.

And thanks for pointing out the wrong code.

1. I also found the problem in line 146 in models.resnet.py: self.conv1.weight.data = torch.mul(self.conv1.weight, self.mask1.weight)

I already modified this line with Autograd class for preserving weight. But I have participated some compression challenge and projects. So It was too busy to update. I will update it as soon as possible.

2. In the paper, author used gradient of masked model for updating original model. For using gradient of masked model, I used masking weights at every forward step.

[bangawayoo]

Thanks!