search

yiwenguo / Dynamic-Network-Surgery

Caffe implementation for dynamic network surgery.
Other
186 stars 70 forks source link

c_rate setting #2

Closed ALHOLA closed 7 years ago

ALHOLA commented 7 years ago

Hello, Is there any algorithm or formula for setting the c_rate value in the cconvolution_param or cinnerproduct_param.I tried out different values and this repo works weird with float c_rate values. The value of 4 in the sample prototxt for lenet doesn't work in my case. Can you help to come up with the right value for c_rate?

yiwenguo commented 7 years ago

In the example for LeNet-5, we didn't really mean that the c_rate values for different layers should all be set as 4. Of course you can try float (and even negative float) c_rate values. Can @ALHOLA provide more details on the weird results? Thanks.

akamaster commented 7 years ago

Provide the right values for c_rate so that results of your NIPS paper can be reproducded. Prototxt file would be even better.

yiwenguo commented 7 years ago

@akamaster, Indeed, the hyper-parameters are not easy to tune. Please be patient. I did some experiments this morning and maybe you can follow. Taking LeNet-5 as an example, you can try something like: {conv1: c_rate (1), iter_stop (14000); conv2: c_rate (3), iter_stop (14000); ip1: c_rate (4), iter_stop (10000); ip2: c_rate (2.5), iter_stop (10000)} {conv1: c_rate (1), iter_stop (14000); conv2: c_rate (3), iter_stop (12000); ip1: c_rate (4), iter_stop (10000); ip2: c_rate (2.5), iter_stop (8000)} {conv1: c_rate (2.5), iter_stop (14000); conv2: c_rate (3), iter_stop (14000); ip1: c_rate (4), iter_stop (14000); ip2: c_rate (2.5), iter_stop (14000)}.