search

ARM-software / sesr

Super-Efficient Super Resolution
Apache License 2.0
89 stars 15 forks source link

Question about the residual #3

Closed eternalding closed 2 years ago

eternalding commented 2 years ago

Greetings. In models/sesr.py, self.linear_blocks were set to have in_filters = out_filters = FLAGS.int_features. Which implied that, when trained with collapsed linear blocks (LinearBlock_c), the residual will be added to the weight tensor wt. (models/model_utils.py ln:139) However, in models/sesr.py ln:72, if self.residualFlag is set to true, it seems that an extra residual will be added to the input. Will this cause any problem?

Thanks for your great work btw. Wenny

kbhardwa commented 2 years ago

Hi Wenny, thanks for the question. The self.residualFlag is set to True only for LinearBlock_e and not for LinearBlock_c. Please see the comment here https://github.com/ARM-software/sesr/blob/master/models/sesr.py#L38-L44. You are correct, if we did two residual adds in case of LinearBlock_c, there may be issues. Only LinearBlock_c has internal residual add and LinearBlock_e does not have this residual. Since we wanted to support both Expanded Linear Block (LinearBlock_e) and Collapsed Linear Block (LinearBlock_c) during training, we used this self.ResidualFlag to support it. Hope this helps.

eternalding commented 2 years ago

Dear kbhardwa, Thanks for your quick reply. That makes everything clear. Best regards, Wenny