Open sgmath12 opened 2 years ago
Figure 3 illustrates the bias in weights at hidden layers with 512 filters, not the classifier. In that figure, we compare the naive model weights to the model with weight decay only, so L2 normalization was not used. Let me know if you have more questions!
If you calculate filter L2 norm on trained model, how did you calculate variance of filter norm ? (On figure 3) Since the model is already trained, the filter is learned, so it is fixed. Thus norm is just a value, without variance. Am I right ?
Thanks.