Calculation of minimal & maximal pruning ratios

Tencent / PocketFlow

An Automatic Model Compression (AutoMC) framework for developing smaller and faster AI applications.

https://pocketflow.github.io

Other

2.78k stars 490 forks source link

Calculation of minimal & maximal pruning ratios #260

Closed Site1997 closed 5 years ago

Site1997 commented 5 years ago

In PocketFlow/learners/weight_sparsification/rl_helper.py:

# obtain the minimal & maximal pruning ratios of each maskable variable
prune_ratio_min = max(0.0, 1.0 - (1.0 - FLAGS.ws_prune_ratio) * 3.0)
prune_ratio_max = 1.0 - (1.0 - FLAGS.ws_prune_ratio) / 3.0

I don't understand what's the logic behind the calculation of ratio_min and ratio_max. Eg, what does "3.0" mean here? Could someone explain it to me? Thanks in advance!

jiaxiang-wu commented 5 years ago

We use prune_ratio_min and prune_ratio_max to limit the range of selected pruning ratio of each layer. "3.0" is a manually tuned hyper-parameter which seems to work fine.