Yes, Pruning is an invasive process. However, if you can find the sweet spot (i.e...) the tradeoff between model degradation and removing redundant data.
We can cram this space, to do so, the package employs a layer-wise sensitivity scan that parses through every layer of the model and finds the sweet spot. Usually, this is quite expensive but the package has one of the fastest ways to find the best pruning ratio.
Thus even after pruning, we make a wise decision to only prune the redundant data possible. Also, fine-tuning is applied post to the pruning such that we can regain the degraded accuracy.
I hope this answers this question. Feel free to open it again if you do not feel satisfied with the answer.
Yes, Pruning is an invasive process. However, if you can find the sweet spot (i.e...) the tradeoff between model degradation and removing redundant data.
We can cram this space, to do so, the package employs a layer-wise sensitivity scan that parses through every layer of the model and finds the sweet spot. Usually, this is quite expensive but the package has one of the fastest ways to find the best pruning ratio.
The tutorial explains this in detail: https://sconce.readthedocs.io/en/latest/tutorials/Pruning.html#lets-first-evaluate-the-accuracy-and-model-size-of-dense-model Look out for the header “Sensitivity Scan”.
Thus even after pruning, we make a wise decision to only prune the redundant data possible. Also, fine-tuning is applied post to the pruning such that we can regain the degraded accuracy.
I hope this answers this question. Feel free to open it again if you do not feel satisfied with the answer.