Why use synthetic HR reference images in training instead of the real ref?

[codyshen0000]

I mean there is only a small alignment between the synthetic HR reference images and GT， can it fit the real reference image with large gaps?

[yumingj]

Hi, for the training of the correspondence network, we use the synthesized reference because by doing so we can have the GT correspondence between the image pairs.

The transformations used for synthesizing image pairs are a range, including small alignments and relatively large alignments.

[codyshen0000]

thx, I see. And another question is that if the reference has relatively large alignments or even is not similar to LR, what will be the result? I found most results in the papers are relatively small alignments.

[yumingj]

If the reference image has extremely large misalignment or even is not similar to LR, the performance would be inevitably degraded. Because in these cases, the reference image itself does not contain too much texture information that could be transferred to aid the reference-based SR. To better handle these challenges, you may try to design more complicated transformations in the training phase.

[codyshen0000]

Got it, thanks a lot!