search

luuyin / OWL

Official Pytorch Implementation of "Outlier Weighed Layerwise Sparsity (OWL): A Missing Secret Sauce for Pruning LLMs to High Sparsity"
https://arxiv.org/pdf/2310.05175.pdf
MIT License
51 stars 8 forks source link

No change in model size after pruning #7

Closed panchal2002 closed 5 months ago

panchal2002 commented 7 months ago

Hey, I wanted to know whether the size of model will be same after pruning or it'll be reduced? I've tried to prune OPT-125M model but the size of the model is same as before which is 250MB. Thanks in advance

123Bailey123 commented 7 months ago

The model will remain the same size since OWL/SparseGPT/Wanda etc., are all unstructured pruners (parameters are just set to zero) - Use something like LLM-Pruner for structured pruning which will reduce the model size (and quality).