search

mit-han-lab / once-for-all

[ICLR 2020] Once for All: Train One Network and Specialize it for Efficient Deployment
https://ofa.mit.edu/
MIT License
1.89k stars 333 forks source link

Question about the calculation of importance(L1 Norm) #50

Open pprp opened 3 years ago

pprp commented 3 years ago

Thank you for your great job.

I have a question about the calculation of importance. Here in Once for all, the importance is calculated by the input dimension.

https://github.com/mit-han-lab/once-for-all/blob/cfa0722d57e3a2391eb36b8cf613dd17ff7a32ae/ofa/imagenet_classification/elastic_nn/modules/dynamic_layers.py#L263

But in Pruning_filters_for_efficient_convnets, the importance is calculated by the output dimension.

https://github.com/tyui592/Pruning_filters_for_efficient_convnets/blob/00ec7b7ae9e8f9bd3973888590728477e73537d9/prune.py#L69

sum_of_kernel = torch.sum(torch.abs(kernel.view(kernel.size(0), -1)), dim=1)

Is there any intrinsic reason to calculated by the input dimension?

Thanks!

tiandunx commented 2 years ago

I'm also very confused about the implementation of importance calculation which is obtained by the input dimension.