Residual network

[accosmin]

• implement a residual network: see https://arxiv.org/pdf/1512.03385.pdf

[accosmin]

• some other variation for convolution networks without padding: 2 convolution layers can be combined with a convolution layer with the mask size the sum of the mask sizes of those 2 layers. For example: (3x3 + act + 5x5) can be combined with (7x7)
• the combination can be generalized: plus (like in the residual networks), cat4 (concatenation of the feature maps), weighting...

[accosmin]

• see also some other variations: https://blog.init.ai/residual-neural-networks-are-an-exciting-area-of-deep-learning-research-acf14f4912e9 https://chatbotslife.com/resnets-highwaynets-and-densenets-oh-my-9bb15918ee32 (highway networks) https://blog.waya.ai/deep-residual-learning-9610bb62c355 http://torch.ch/blog/2016/02/04/resnets.html https://codesachin.wordpress.com/2017/02/19/residual-neural-networks-as-ensembles/