search

rahulvigneswaran / Lottery-Ticket-Hypothesis-in-Pytorch

This repository contains a Pytorch implementation of the paper "The Lottery Ticket Hypothesis: Finding Sparse, Trainable Neural Networks" by Jonathan Frankle and Michael Carbin that can be easily adapted to any model/dataset.
325 stars 91 forks source link

Potential bug #28

Open adnan1306 opened 1 year ago

adnan1306 commented 1 year ago

In line286 of main.py, it should be: new_mask = np.where(abs(tensor) > percentile_value, 0, mask[step]) so that smaller weights are set to zero.

https://github.com/rahulvigneswaran/Lottery-Ticket-Hypothesis-in-Pytorch/blob/34a8c9678406a1c7dd0fec4c9f0d25d017be55fb/main.py#L286

wanxinran commented 1 year ago

No the code is correct, check the np.where docs. 0 is set when tensor is smaller than the cutoff value.