search

pytorch / ao

PyTorch native quantization and sparsity for training and inference
BSD 3-Clause "New" or "Revised" License
725 stars 92 forks source link

What is the difference between WeightNormSparsifier and torch.nn.utils.prune.l1_unstructured ? #724

Open mayank64ce opened 3 weeks ago

supriyar commented 3 weeks ago

cc @jcaip

jcaip commented 3 weeks ago

Hey @mayank64ce torch.nn.tils.prun.l1_unstructured is no longer maintained, so I would recommend using the WeightNormSparsifier. The sparsifier also allows for more configs, like block_size, or intra block sparsity.

Functionally however, they do the same thing - mask dense models to put zeros in the "right" place.

mayank64ce commented 3 weeks ago

I see, in the WeightNormSparsifier, there seem to be some additional parameters other different from l1_unstructured. How can I simulate the same behavior for a give prune_ratio .

Also can this sparse_block_shape parameter be of any arbitrary shape ?