search

ethanhe42 / channel-pruning

Channel Pruning for Accelerating Very Deep Neural Networks (ICCV'17)
https://arxiv.org/abs/1707.06168
MIT License
1.08k stars 310 forks source link

How to calculate the rank? #47

Closed kakusikun closed 6 years ago

kakusikun commented 7 years ago

https://github.com/yihui-he/channel-pruning/blob/master/lib/net.py#L1301

The rankdic is set beforehand and it looks like only for VGG16.

it seems that any formula or theorem about calculating rankdic does not mention in paper.

is it the experimental outcome?

if true, what's the rank for ResNet-50? Since the rankdic in the release source code of ResNet-50 is same as VGG16 https://github.com/yihui-he/channel-pruning/releases/tag/ResNet-50-2X

thanks.

gaqiness commented 7 years ago

I have the same enquiries too...

FatherOfHam commented 7 years ago

I guess we can deduce that from the protocol of resnet-50-2X, but I have the same enquiries too.

slothkong commented 7 years ago

Maybe this could help: https://github.com/yihui-he/channel-pruning/wiki/ResNet-perserving-ratio

kakusikun commented 7 years ago

@slothkong Is it for spatial decomposition? Does it help you figure out calculating the rank?

slothkong commented 7 years ago

@MaKiTann I dont know exactly what spatial decomposition is. Maybe is best if you refer to the paper. I'm not sure how it affect rank either, but there seems to be a relation...

kakusikun commented 7 years ago

@slothkong According to this paper, spatial decomposition (https://arxiv.org/abs/1405.3866) is one of the 3C method mentioned in the README. Spatial decomposition use low rank to approximate the filter, therefore, how to decide the rank is apparently important. But the authors in spatial decomposition and this paper seems that do not show how to decide it and just set it in the code.

slothkong commented 7 years ago

@MaKiTann That is 100% correct my friend. Is usually like that in papers about pruning algorithms. I'm trying to run test in different models, with different datasets (using channel pruning only ) to kind of get a big-picture view of what are the tolerable pruning degrees, if I manage to get this code to execute pruning only that is :)

kakusikun commented 7 years ago

@slothkong This is a similar method and question. https://github.com/chengtaipu/lowrankcnn/blob/master/imagenet/models_vgg/config.json

slothkong commented 7 years ago

@MaKiTann , I just found this: https://github.com/yihui-he/channel-pruning/blob/3a4614de8c11aeb2ab981ddc619927c0e60a6577/lib/net.py#L1505-L1507 (ㅇㅅㅇ)

kakusikun commented 6 years ago

@slothkong Sorry, last week was a busy week. It looks like that it's meant to be hide something ....

ethanhe42 commented 6 years ago

Determining rank is out of our paper's scope. Ranks are manually set. In channel decomposition, there's a heuristic method proposed.