A study that evaluated methods that prune from initialization. They are better than random, but consistently perform worse than methods that prune after training. In addition, shuffling weights within each layer or reinitializing weights results in equal or better accuracy rather than degradation. Furthermore, these can be replaced by a method that determines the rate of weight pruning per layer rather than the rate of weights to be pruned
TL;DR
A study that evaluated methods that prune from initialization. They are better than random, but consistently perform worse than methods that prune after training. In addition, shuffling weights within each layer or reinitializing weights results in equal or better accuracy rather than degradation. Furthermore, these can be replaced by a method that determines the rate of weight pruning per layer rather than the rate of weights to be pruned
Why it matters:
Paper URL
https://arxiv.org/abs/2009.08576
Submission Dates(yyyy/mm/dd)
2020/09/18
Authors and institutions
Jonathan Frankle, Gintare Karolina Dziugaite, Daniel M. Roy, Michael Carbin
.
Methods
Results
Comments