Closed cheng221 closed 10 months ago
This is called Skip Init and it helps stabilize training. (https://paperswithcode.com/method/skipinit)
This is called Skip Init and it helps stabilize training. (https://paperswithcode.com/method/skipinit)
Thank you kindly for your answer, it has been incredibly helpful to me. Have a good day!
Hello! Thanks for your great work advancing the progress of image restoration. Recently, I conducted some experiments about the skip connection of NAFBlock. Because the form of skip connection in code is
y = inp + x * self.beta
, which has additional parameters control the output of the convolutional branch. And when replacingy = inp + x * self.beta
withy = inp + x
, the performance will drop by about 0.2 PSNR in my settings. So I want to know why it affects the performance. Finally, are there some kinds of literature or blogs discussing this design? Thanks for taking your valuable time to answer my question!