Question about skip-connection in Reconstruction Task

[babbu3682]

I think the role of skip-connection is rather disturbing in your paper where reconstruction tasks are usually performed, what do you think?

Using skip connection in reconstruction often transfers a lot of information intact in the initial layer. However, you put a strong distortion on input, probably which is why the cheating problem does not occur.

what do you think about it? Does skip-connection play an important role in your reconstruction task?

[MrGiovanni]

Hi @babbu3682

Skip connection itself does not play an important role in the reconstruction task and sometimes gives a trivial solution if the input and output are too similar. Referring to Sec. 2 (Removing Skip Connection) in

• Feng, R., Zhou, Z., Gotway, M. B., & Liang, J. (2020). Parts2Whole: Self-supervised Contrastive Learning via Reconstruction. In Domain Adaptation and Representation Transfer, and Distributed and Collaborative Learning (pp. 85-95). Springer, Cham.

Skip connections have been shown to be very important for target segmentation tasks. If the proxy task has no skip connection and the target task has skip connections in the model, only the encoder can be used for transfer learning; the pre-trained decoder has to be dismissed due to the inconsistency of skip connections. We kept the skip connection in Models Genesis to make use of the pre-trained decoder for target segmentation tasks (not really considering the reconstruction task).

Thanks,

Zongwei

[MrGiovanni]

Additionally, skip connections might be helpful for the purpose of anomaly detection. Referring to Sec. 5.6 in the Models Genesis paper: "Could Models Genesis detect infected regions from images autonomously?" In this case, it is important to preserve texture/structural details in the input images.