Say we train a FC3 model on some problem and get layer criticalness c1,c2,c3 (for layers 1,2,3 resp.).
Now, say we freeze every layer except layer i and reset layer i to epoch 0, and re-train the model (train only layer i). Only train until we reach similar accuracy to the original trained model (until layer i is 'healed'), and take note of the number of epochs ei required to heal.
Is there any correlation between ei and ci for i=1,2,3?
Robust layers obviously have very small ei values because even reset to epoch 0 they still yield good results, but for networks with 'spread out' criticalness it would be interesting if we could find a pattern.
Say we train a FC3 model on some problem and get layer criticalness
c1,c2,c3
(for layers1,2,3
resp.). Now, say we freeze every layer except layeri
and reset layeri
to epoch 0, and re-train the model (train only layeri
). Only train until we reach similar accuracy to the original trained model (until layeri
is 'healed'), and take note of the number of epochsei
required to heal.Is there any correlation between
ei
andci
fori=1,2,3
?Robust layers obviously have very small
ei
values because even reset to epoch 0 they still yield good results, but for networks with 'spread out' criticalness it would be interesting if we could find a pattern.Check this on VGG networks