Closed YongboLiang closed 5 years ago
In all our reported experiments, res scale is set to 1.0 by default. Thus this trick is not reported in our factsheet.
Although the res scale trick is wield, I guess it could help when your SR network is extremely large. Another reference is the paper about training ImageNet in one hour in which authors claim that setting the scale of last BN in a residual block to zero helps overall performance.
@JiahuiYu Thanks for your reply. The value of n_feats and block_feats are 32 and 128 in your experiment and you use res_scale with 1.0. when the n_feats in EDSR is 256 , they set res_scale to be 0.1. Compared with EDSR, I plan to set n_feats and block_feats to be 128 and 512. Then do you have advice about res_scale when the value of n_feats and block_feats are so big? Best~~
If you use relatively reasonable depth, eg., 16 residual blocks, using 1.0 should be fine. If the depth is crazy like 32 or 64, probably you will need to use 0.1. My experience is that using res scale only brings slight performsnce gain.
It is a good question though, and worth investigating. The trick of res scale is a mystery so far.
got it~~ thanks~~
When the number of feature maps increase above a certain level, the author will set residual scaling with 0.1 to ensure the training procedure stable in EDSR. I don't find the illustration about res_scale in your paper while I find that in your code. Then I want to know how do you set res_scale base on the value of n_feats and block_feats? Best~