search

jay-mahadeokar / pynetbuilder

pyNetBuilder is a modular pytonic interface with builtin modules for generating popular caffe prototxt network file definitions.
BSD 2-Clause "Simplified" License
328 stars 140 forks source link

scale layer in Resnet50 #3

Open unlabeledData opened 8 years ago

unlabeledData commented 8 years ago

Someone has said that the scale_param value should be set as 1.0000001 so that the network will converge. But in your resnet50 model, the scale_param is not be defined. Is is not impossible for the resnet50 model? scale_param { axis: 1 num_axes: 1 filler { type: "constant" value: 1.0000001 }

jay-mahadeokar commented 8 years ago

Who is someone? :-) I believe scale_param filler is 1 by default see this Refer this discussion on how Scale layer is used with BatchNorm.

I am able to properly converge using the existing prototxt params.

jay-mahadeokar commented 8 years ago

@airRobotCool , closing this issue. Please feel free to reopen if you have any other question.

unlabeledData commented 8 years ago

Thank you very much for your great work. You are right, I have made your networks converged on imagenet. The scale_param value is set as 1.0000001 by ducha-aiki on https://github.com/ducha-aiki/caffe/blob/elu/examples/BN-nator.ipynb

jay-mahadeokar commented 8 years ago

Glad you found it useful! If you use it to generate any variants of residual networks, feel free to contribute the models back via PR.