Closed yellowisher closed 5 years ago
They are somewhat equivalent: conv{ab}(concat(a,b)) is actually equivalent of conv{a}(a)+conv{b}(b), when the weights of conv{ab} are the weights of conv{a} and conv{b} concatenated.
Oh I just figured it out. Thanks for the answer.
It seems that previous tensors are added with convolution operation in shortcut connection. But in U-Net(which you referenced), shortcut connection contains no convolution but only concatenation. Is there any reason or source for using
conv+add
overconcat
or justadd
?