search

anonymous47823493 / EagleEye

(ECCV'2020 Oral)EagleEye: Fast Sub-net Evaluation for Efficient Neural Network Pruning
304 stars 68 forks source link

a question about adaptive bn? #29

Closed freeman-1995 closed 3 years ago

freeman-1995 commented 3 years ago

HI , @anonymous47823493

After reading eagle eye, bn-adaptive makes the accuracy on val_dataset more reliable to represent the pruned network as a good network after finetuning. but I have a question about adaptive-BN technology. this technique has already been proposed previously, some jobs like bignas, slimmable network..., they renamed this technique as bn calibration. so I just can't get the innovation.

bezorro commented 3 years ago

Hi @freeman-1995, we appreciate your professional review.

We agree adjusting-BN is not a new idea. Itserved for many non-pruning purposes in existing works. As the title of our paper is 'Fast Subnet Evaluation', we believe the novelty of our work is more of introducing a correlation analysis into the domain of pruning algorithm, which results in a simple, generic yet efficient pruning approach using adaptive-BN.

Such correlation study unveils a problematic practice that is widely found but barely emphasized in many recent works in this area: using inap-propriate BN statistics to guide pruning can be misleading[1][2]. Without highlighting, this issue is likely to be con-tinuously found in coming works in this domain such as [3].

[1] Pruning filters for efficient convnets. [2] Amc: Automl for model compression and acceleration on mobile devices. [3] AutoCompress: An Automatic DNN Structured Pruning Framework for Ultra-High Compression Rates.