New experiments with resnet 50 elu nobn:
res_ror -- is ror[1] like extended connection with blocks, but standart RoR have projections (with blocks) 1->2 2->3 3->4 4->5 1->5. In this experiments different connection havent 1->5, but have 1->3 2->3 2->4 2->5 3->4 3->5 4->5
res_ror_1x3 -- some blocks sre B(1,3,1) some blocks B(1x3,3x1) and 1x3xdim/4 have dim/4 like in B(1,3,1)
res_ror_1x5 -- B(1,1x5,5x1,1)
res_ror_var -- have different blocks
Residual Networks of Residual Networks: Multilevel Residual Networks
New experiments with resnet 50 elu nobn: res_ror -- is ror[1] like extended connection with blocks, but standart RoR have projections (with blocks) 1->2 2->3 3->4 4->5 1->5. In this experiments different connection havent 1->5, but have 1->3 2->3 2->4 2->5 3->4 3->5 4->5 res_ror_1x3 -- some blocks sre B(1,3,1) some blocks B(1x3,3x1) and 1x3xdim/4 have dim/4 like in B(1,3,1) res_ror_1x5 -- B(1,1x5,5x1,1) res_ror_var -- have different blocks