search

GUOShuxuan / expandnets

Official implementation of ExpandNets (NeurIPS2020)
https://proceedings.neurips.cc/paper/2020/hash/0e1ebad68af7f0ae4830b7ac92bc3c6f-Abstract.html
BSD 3-Clause "New" or "Revised" License
36 stars 6 forks source link

Compute expansion from existing network #1

Open shachargluska opened 3 years ago

shachargluska commented 3 years ago

Hi, I find your work really interesting but I have a question. How can I expand a network with existing weights? I want to train a network using this scheme, but I need to start from existing weights, such as imagenet pretraining.

Adding 1x1 convolution without changing the model is simple. So Expand-CL can be implemented using existing weights.

I'm not sure how to do Expand-CK. If I have an existing k X k kernel for k > 3, how do I split it into a 3x3 kernels?

GUOShuxuan commented 3 years ago

Hi,

Thank you for your interest. Currently, our method supports training ExpandNets from scratch. If you want to use pre-trained weights W to initialize W1, W2 where W=W1*W2 when using Expand-CK, I think you have to solve the matrix equations with your own constraints. Otherwise, there will be multiple solutions.

Cheers, Shuxuan