Why only 'fast_weights(in_weights)' are updated during meta-learning stage?

[raidria]

I wonder why only the 'fast_weights(in_weights)' are updated excluding conv&deconv layers in inner loop. (line 166 in main.py)

Is it make the learning faster or something?

[WellyZhang]

It's basically for efficiency.

On Fri, Oct 8, 2021 at 2:04 PM raidria @.***> wrote:

I wonder why only the 'fast_weights(in_weights)' are updated excluding conv&deconv layers in inner loop. (line 166 in main.py)

Is it make the learning faster or something?

— You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub https://github.com/WellyZhang/MetaStyle/issues/4, or unsubscribe https://github.com/notifications/unsubscribe-auth/ACWYBKVISTUVCOL7FRXFVRLUF2CYPANCNFSM5FS2ZN5Q . Triage notifications on the go with GitHub Mobile for iOS https://apps.apple.com/app/apple-store/id1477376905?ct=notification-email&mt=8&pt=524675 or Android https://play.google.com/store/apps/details?id=com.github.android&referrer=utm_campaign%3Dnotification-email%26utm_medium%3Demail%26utm_source%3Dgithub.

[sunwoo76]

@WellyZhang I have same question. Is there any reference which explains this tricks? Could you give brief or little explanation it? Thanks :)

[WellyZhang]

There are two reasons:

1. An earlier paper shows that the conditional instance normalization alone suffices for style transfer. See https://arxiv.org/abs/1610.07629
2. Back then when we had limited compute, we could not afford the memory consumption for updating the entire network.

On Wed, Oct 13, 2021 at 1:23 AM sunshower76 @.***> wrote:

@WellyZhang https://github.com/WellyZhang I have same question. Is there any reference which explains this tricks? Could you give brief or little explanation it? Thanks :)

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/WellyZhang/MetaStyle/issues/4#issuecomment-941215498, or unsubscribe https://github.com/notifications/unsubscribe-auth/ACWYBKT6PIKXC7XOHEBKKNLUGRVHZANCNFSM5FS2ZN5Q . Triage notifications on the go with GitHub Mobile for iOS https://apps.apple.com/app/apple-store/id1477376905?ct=notification-email&mt=8&pt=524675 or Android https://play.google.com/store/apps/details?id=com.github.android&referrer=utm_campaign%3Dnotification-email%26utm_medium%3Demail%26utm_source%3Dgithub.