torch.add in Unext module?

[Capchenxi]

Hello, I saw in the source code, you used torch.cat() to combine the downsampled features with the upsampled features? Is any specific reason for that operation? Is it because using torch.add() has less operations than torch.concate() when the network process forward? Since I remember in unet the downsampled features and the upsampled features are concatenated together rather than add. Thanks.

[jeya-maria-jose]

Hi, yeah, using add reduces complexity. Addition can be considered as non weighted concatenation. Please check this out: https://stats.stackexchange.com/questions/361018/when-to-add-layers-and-when-to-concatenate-in-neural-networks

[Capchenxi]

Hi, yeah, using add reduces complexity. Addition can be considered as non weighted concatenation. Please check this out: https://stats.stackexchange.com/questions/361018/when-to-add-layers-and-when-to-concatenate-in-neural-networks

Thanks for your prompt reply. That makes sense to me.