A issue about performance gain via weight normalization

[Vandermode]

Hi, I read through the paper, and find the discussion of weight normalization as follow:

It is also noteworthy that naively introducing weight normalization in training image restoration networks may not help that much. We find empirically that weight normalization allows higher learning rate (i.e. 10×), with which the loss of training normal networks explodes. The advantages of weight normalization are shown in Figure 3.

It seems naively introducing the weight normalization would not help much towards super-resolution task, but in Fig. 3 we definitely observe performance gain brought by weight normalization. So I was wondering how about non-naive way to introduce wn in the paper.

[JiahuiYu]

@Vandermode Thanks for your interest. The describe is not accurate actually. Weight normalization does improve performance, but only when you train with a higher learning rate.

[JiahuiYu]

@Vandermode We have updated the report/factsheet. Hope it helps.