search

KarenUllrich / Tutorial_BayesianCompressionForDL

A tutorial on "Bayesian Compression for Deep Learning" published at NIPS (2017).
MIT License
203 stars 48 forks source link

Do you have any tricks to choose threshold of every layer? #4

Closed larenzhang closed 5 years ago

larenzhang commented 6 years ago

After the model have been trained by the group horseshoe with the half-cauchy scale priors,I want to pruning the parameters.But I find it's hard to find the pruning threshold. Do you have any trick to pick up the threshold?Thanks very much. @KarenUllrich

KarenUllrich commented 6 years ago

Hi larenzhang,

according to our experience, the threshold is not a sensitive hyperparameter. Our approach was to look at a plot of the distribution and just choose one that seems to separate the modes well.

Cheers Karen