Split `Wanda` and `SparseGPT`

neuralmagic / sparseml

Libraries for applying sparsification recipes to neural networks with a few lines of code, enabling faster and smaller models

Apache License 2.0

2.07k stars 148 forks source link

Split `Wanda` and `SparseGPT` #2266

Closed rahul-tuli closed 6 months ago

rahul-tuli commented 6 months ago

This PR refactors SparseGPT modifier(s) to remove their inheritance from Wanda, making SparseGPT fully independent. The change improves the modularity of the SparseGPT architecture, making it more straightforward to maintain and extend/modify

Test plan: Ran all the tests, + a manual one shot script on HuggingFaceM4/tiny-random-LlamaForCausalLM