mobilefacenet剪枝效果差

[eeric]

https://github.com/deepinsight/insightface/blob/master/recognition/arcface_torch/backbones/mobilefacenet.py

模型定义 https://github.com/deepinsight/insightface/blob/master/recognition/arcface_torch/backbones/mobilefacenet.py

使用group_norm剪枝方式，参数如下： model_name = 'mobilefacev2' global_pruning = 'True' reg = 1e-4 max_ch_sparsity = 1.0 target_flops = 1.0 prune_pm = 'True' method = "group_norm"

使用insightface训练框架，训练20轮，效果比原来差很多

[VainF]

您好，global pruning=True有过度剪枝的风险，尤其是在没有稀疏训练的情况下。请尝试一下打印剪枝后模型看一看是否有一些层被过度剪枝了，此时可以尝试global_pruning=False。

[eeric]

好，我试试，另外，作者本人有空也请试验剪枝这种模型，先谢谢了。

[eeric]

原模型结构片段 (3): DepthWise( (layers): Sequential( (0): ConvBlock( (layers): Sequential( (0): Conv2d(640, 256, kernel_size=(1, 1), stride=(1, 1), bias=False) (1): BatchNorm2d(256, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True) (2): FReLU( (conv_frelu): Conv2d(256, 256, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), groups=256) (bn_frelu): BatchNorm2d(256, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True) ) ) ) (1): ConvBlock( (layers): Sequential( (0): Conv2d(256, 256, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), groups=256, bias=False) (1): BatchNorm2d(256, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True) (2): FReLU( (conv_frelu): Conv2d(256, 256, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), groups=256) (bn_frelu): BatchNorm2d(256, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True) ) ) ) (2): LinearBlock( (layers): Sequential( (0): Conv2d(256, 640, kernel_size=(1, 1), stride=(1, 1), bias=False) (1): BatchNorm2d(640, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True) ) 采用上面剪枝参数 (3): DepthWise( (layers): Sequential( (0): ConvBlock( (layers): Sequential( (0): Conv2d(13, 8, kernel_size=(1, 1), stride=(1, 1), bias=False) (1): BatchNorm2d(8, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True) (2): FReLU( (conv_frelu): Conv2d(8, 8, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), groups=8) (bn_frelu): BatchNorm2d(8, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True) ) ) ) (1): ConvBlock( (layers): Sequential( (0): Conv2d(8, 8, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), groups=8, bias=False) (1): BatchNorm2d(8, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True) (2): FReLU( (conv_frelu): Conv2d(8, 8, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), groups=8) (bn_frelu): BatchNorm2d(8, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True) ) ) ) (2): LinearBlock( (layers): Sequential( (0): Conv2d(8, 13, kernel_size=(1, 1), stride=(1, 1), bias=False) (1): BatchNorm2d(13, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True) )

对比来看，结构和层数没变，就是通道数减少太多。 另外，把global_pruning = False，也和True剪枝结构一样的。

[VainF]

这应该是剪枝过度了… 可以尝试调低一点剪枝率。

[eeric]

好，我试试

[eeric]

@VainF 使用group_norm剪枝方式，参数如下： global_pruning = 'True' #True reg = 5e-4 iterative_steps = 40 #40 max_ch_sparsity = 0.5 speed_up = 2.55 prune_pm = 'True' method = 'group_norm'

只剪枝不用稀疏，效果能差一些，大概frr@farle-5变差1倍 如果先稀疏训练，再剪枝训练，效果更差一些。 有没有好的建议？

[Flyingpige]

https://github.com/deepinsight/insightface/blob/master/recognition/arcface_torch/backbones/mobilefacenet.py

模型定义 https://github.com/deepinsight/insightface/blob/master/recognition/arcface_torch/backbones/mobilefacenet.py

使用group_norm剪枝方式，参数如下： model_name = 'mobilefacev2' global_pruning = 'True' reg = 1e-4 max_ch_sparsity = 1.0 target_flops = 1.0 prune_pm = 'True' method = "group_norm"

使用insightface训练框架，训练20轮，效果比原来差很多

我目前在使用该框架对我改进的ShuffleNet进行剪枝，按照作者知乎上的教程，但是出现了进程卡死的bug，请问您能让我参考一下您的对模型进行剪枝的代码吗？感激不尽~

[study-clever]

https://github.com/deepinsight/insightface/blob/master/recognition/arcface_torch/backbones/mobilefacenet.py 模型定义 https://github.com/deepinsight/insightface/blob/master/recognition/arcface_torch/backbones/mobilefacenet.py 使用group_norm剪枝方式，参数如下： model_name = 'mobilefacev2' global_pruning = 'True' reg = 1e-4 max_ch_sparsity = 1.0 target_flops = 1.0 prune_pm = 'True' method = “group_norm” 使用insightface训练框架，训练20轮，效果比原来差很多

我目前在使用该框架对我改进的ShuffleNet进行剪枝，按照作者知乎上的教程，但是出现了进程卡死的bug，请问您能让我参考一下您的对模型进行剪枝的代码吗？感激不尽~

请问您实现了对shuffleNetV2的剪枝了吗？