Say we train a FC3 model on some problem and get layer criticalness c1,c2,c3 (for layers 1,2,3 resp.).
Now, say we freeze every layer except layer i and reset layer i to epoch 0, and re-train the model (train only layer i). Only train until we reach similar accuracy to the original trained model (until layer i is 'healed'), and take note of the number of epochs ei required to heal.
Is there any correlation between ei and ci for i=1,2,3?
Robust layers obviously have very small ei values because even reset to epoch 0 they still yield good results, but for networks with 'spread out' criticalness it would be interesting if we could find a pattern.
Say we train a FC3 model on some problem and get layer criticalness
c1,c2,c3(for layers1,2,3resp.). Now, say we freeze every layer except layeriand reset layerito epoch 0, and re-train the model (train only layeri). Only train until we reach similar accuracy to the original trained model (until layeriis 'healed'), and take note of the number of epochseirequired to heal.Is there any correlation between
eiandcifori=1,2,3?Robust layers obviously have very small
eivalues because even reset to epoch 0 they still yield good results, but for networks with 'spread out' criticalness it would be interesting if we could find a pattern.Check this on VGG networks