search

vlfeat / matconvnet

MatConvNet: CNNs for MATLAB
Other
1.4k stars 753 forks source link

Does we have any function to realize weight sharing? #114

Closed llyydd007 closed 8 years ago

llyydd007 commented 9 years ago

Hello,everyone. Does matconvnet has any ways to realize weight sharing? Thanks very much!

vedaldi commented 9 years ago

Hi, it is not implemented directly, but it can be done by designing a new wrapper or modifying the existing one. However, it is likely that we will add this functionality in the future.

On 6 Apr 2015, at 14:09, llyydd007 notifications@github.com wrote:

Hello,everyone. Does matconvnet has any ways to realize weight sharing? Thanks very much!

— Reply to this email directly or view it on GitHub https://github.com/vlfeat/matconvnet/issues/114.

llyydd007 commented 9 years ago

Ok, thank you. I am looking forward for it. However, I wonder, if I have N groups and I want these N groups have a sharing weights,if I can do the forward separately with the convolutional layers fixed(filter fixed) and then calculated the gradients separately with the convolutional layers fixed,then I calculated the gradient as the average gradient and then I do the backward for the each group. It seems in this way,I can modified the matlab codes and avoid modified the .cu. Are any mistakes in this process? Thank you!@vedaldi @lenck Best wishes.

lenck commented 9 years ago

Hi, sorry for late response, but if I understand you correctly that should be fine :) Only thing is that in the mex files, the weights are not changed at all - so basically what you need to rewrite is only the part of cnn_train where you update weights (e.g. take the average over the groups' dzdw and apply to the group filters, so all the groups are equal).

llyydd007 commented 9 years ago

Thank you @lenck . I finally just not average over the groups' dzdw but use the the groups' dzdw sequentially to update the groups filter, It seems to go well and It seems to be equal with sum the gradients without averaging. I will try to average it to make a comparison later.

essalim commented 9 years ago

Hello, i am wondering what do you mean by parameter sharing is not implemented directly ? it seems to me that the convolution layers implementation assume the parameter sharing which means every feature map learns the same set of filter. thank you

lenck commented 8 years ago

Hi, weight sharing is now easy to achieve with the DaG wrapper (simply by naming providing two conv layers the same name of the parameters). I hope it is fine that I will close this issue then.

yushuinanrong commented 8 years ago

@lenck Hi Lenck,

Is it possible to share a ratio of weights at each conv layers in MatConvNet?

Thanks