search

liuzhuang13 / DenseNet

Densely Connected Convolutional Networks, In CVPR 2017 (Best Paper Award).
BSD 3-Clause "New" or "Revised" License
4.71k stars 1.07k forks source link

Add layer bugs #1

Closed offbit closed 8 years ago

offbit commented 8 years ago

Hi,

check out the code at

https://github.com/liuzhuang13/DenseNet/blob/master/densenet.lua#L54

param requires nOutChannels, but you pass growth rate.

Also add layer only adds direct connection to the next layer, so you form n-1 connections, as far as I understood from your paper it should have n-2, n-3 , etc... for every layer in the block. I'm guessing this is not the version you used to train the models.

Thanks for sharing.

liuzhuang13 commented 8 years ago

Thanks for opening this issue. The function addLayer() does require nOutChannels, and we use this param to denote number of newly learned feature maps at this layer, which is equal to the growth rate.

Superficially, there's only n-1 connections, but in function addLayer(), every time the new feature maps learned by this layer, concated with all previous layers' feature maps, are passed to subsequent layers, through the use of nn.Identity() and nn.Concat(). It's equivalent to the connection pattern described in paper. This is for ease of implementation, one can also keep a set of each layer's feature maps and every time first concate them then do convolution, but that's a little more compilcated to implement.

This may be a question for many people, thanks for pointing out!

offbit commented 8 years ago

Excellent :) I think I didn't pay much attention, and had in mind the residual connection type. But yes, concat makes perfect sense!

Thanks for the clarifications.