Does the gradient branch participate in backpropagation?

[Feihong-cc]

Thanks for your work! I am confused , does the gradient branch participate in backpropagation?

[Maclory]

Yes, it does~

[Feihong-cc]

oh，but I saw the requires_grad = False of weight_h and weight_v written in the /model/SPSR_model.py code, why is that?

[Maclory]

We use a convolutional layer with fixed kernels to extract gradient maps, which is the reason why we set require_grad=False for weight_h and weight_v. However, the parameters of the gradient branch are updated by back-propagation as normal.

[Feihong-cc]

Therefore, in the process of back propagation, the pixel value of image is optimized rather than the weight of the filter?

[Maclory]

Yes.

[Feihong-cc]

Thank you very much! I also want to know that the gradient branch is passed into the SR branch using torch.cat((grad,sr),dim=1), what is the difference between this (grad+sr)?

[Maclory]

The former is concatenation in dimensions while the latter is addition in values.

[Feihong-cc]

oh, no~ I mean to use torch.cat((grad,sr),dim=1) and (grad+sr) and then send it to the convolutional layer. What is the difference?

[Maclory]

I think concatenation is better since it may contain the situation of addition. However, I think both are reasonable and you can have a try to see which is better.

[Feihong-cc]

Great! thank you for your reply ！My idea is that the gradient contains the high frequency information of the image. After the gradient branch completed, we should be added the output gradient to SR, which is used the (grad+SR) operation, to enhance the high frequency information of sr , so restoring the sharpness of the image. As you said, the concatenation may contain the situation of addition, it's okay to use the concatenation.

[Maclory]

:-)

[hyperparameters]

@Florrie-Giu did you try with addition (grad+SR), I was also thinking in a similar direction. Adding will increase the high freq details and thus improve the sharpness