Open dorimedini opened 5 years ago
Default for Dense
layers is glorot_uniform sampling. As for VGG16 (and VGG19 just below it) the docs just say '...weights=None
(random initialization)...' so I dunno about VGG but there's no observed problem in VGG so let's stick with FC nets.
glorot_uniform is uniform in [-X,X]
where X = sqrt(6 / (fan_in + fan_out))
(total incoming + outgoing edges in the layer). Should probably multiply by this while pruning
Results from #102 indicate layer weights aren't all uniformly sampled from the same interval (otherwise the percent pruned should be constant across layers). Figure out what's going on, and take it into account when pruning!