search

dorimedini / robustness_properties

GLOBIGLOBIGLOBI
0 stars 0 forks source link

Check how weights are sampled, update pruning #104

Open dorimedini opened 5 years ago

dorimedini commented 5 years ago

Results from #102 indicate layer weights aren't all uniformly sampled from the same interval (otherwise the percent pruned should be constant across layers). Figure out what's going on, and take it into account when pruning!

dorimedini commented 5 years ago

Default for Dense layers is glorot_uniform sampling. As for VGG16 (and VGG19 just below it) the docs just say '...weights=None (random initialization)...' so I dunno about VGG but there's no observed problem in VGG so let's stick with FC nets.

dorimedini commented 5 years ago

glorot_uniform is uniform in [-X,X] where X = sqrt(6 / (fan_in + fan_out)) (total incoming + outgoing edges in the layer). Should probably multiply by this while pruning