Can't reproduce network slimming on ImageNet

[huangbiubiu]

We trained the network slimming model with the command https://github.com/Eric-mingjie/rethinking-network-pruning/blob/master/imagenet/network-slimming/README.md#train-with-sparsity, and prune with 50%. However, we could not prune the same result as models you provided.

More specifically, in our result, the classifier.1.weight was pruned to 0 channels, and the classifier.4.weight almost keeps all original channels.

Pruning result:

layer index: 4   total channel: 64       remaining channel: 26
layer index: 8   total channel: 128      remaining channel: 86
layer index: 12          total channel: 256      remaining channel: 111
layer index: 15          total channel: 256      remaining channel: 182
layer index: 19          total channel: 512      remaining channel: 171
layer index: 22          total channel: 512      remaining channel: 176
layer index: 26          total channel: 512      remaining channel: 295
layer index: 29          total channel: 512      remaining channel: 328
layer index: 34          total channel: 4096     remaining channel: 0
layer index: 37          total channel: 4096     remaining channel: 4096

[Eric-mingjie]

Hi @huangbiubiu , thanks for your interest in our work! Here are my suggestions:

1. Can you analyse the distribution of scaling factors in BN of your trained models? Like Figure 4 in the network slimming paper? Normally, network slimming would work by making the scaling factors of BN layers sparse.
2. Maybe you can do the pruning on the Convolution layers and FC layers separately. You can get a 50% pruned models in this way.