search

junyanz / interactive-deep-colorization

Deep learning software for colorizing black and white images with a few clicks.
https://richzhang.github.io/ideepcolor/
MIT License
2.69k stars 447 forks source link

About shortcut connection #55

Closed yellowisher closed 5 years ago

yellowisher commented 5 years ago

It seems that previous tensors are added with convolution operation in shortcut connection. But in U-Net(which you referenced), shortcut connection contains no convolution but only concatenation. Is there any reason or source for using conv+add over concat or just add? 

# Conv7    
model8up = [nn.ConvTranspose2d(512, 256, kernel_size=4, stride=2, padding=1, bias=use_bias)]
model3short8 = [nn.Conv2d(256, 256, kernel_size=3, stride=1, padding=1, bias=use_bias), ]
.
.
.
conv8_up = self.model8up(conv7_3) + self.model3short8(conv3_3)
richzhang commented 5 years ago

They are somewhat equivalent: conv{ab}(concat(a,b)) is actually equivalent of conv{a}(a)+conv{b}(b), when the weights of conv{ab} are the weights of conv{a} and conv{b} concatenated.

yellowisher commented 5 years ago

Oh I just figured it out. Thanks for the answer.