search

JiahuiYu / slimmable_networks

Slimmable Networks, AutoSlim, and Beyond, ICLR 2019, and ICCV 2019
Other
914 stars 131 forks source link

why use slimmable_training instead of universally_slimmable_training: True in Autoslim #38

Closed GYxiaOH closed 4 years ago

GYxiaOH commented 4 years ago

just like title,in my opinion, if you want slim you should know error of model in any channels

JiahuiYu commented 4 years ago

Please look at branch v3.0.0

The master version code for AutoSlim is for inference only. Training code is under branch v3.0.0

GYxiaOH commented 4 years ago

@JiahuiYu yes,i see branch v3.0.0 ,in your autoslim config ,you use slimmable_training but not slimmable_training,and use SwitchableBatchNorm2d SlimmableConv2d, SlimmableLinear,and in my opinion, although calibrate_bn = true

JiahuiYu commented 4 years ago

@GYxiaOH Ah, you are right. The released config is for validation of model performance (inference ). To train a weight-shared model for NAS, you will need to enable universally_slimmable_training.

GYxiaOH commented 4 years ago

thank you for your reply, in my opinion ,i can use your us config to train a model and use slimming to find a best model(after prunning)? by the way ,i see not getattr(m, 'linked', False) in slimming function，it's mean residual_connection in us_mobilenet_v2.py?

JiahuiYu commented 4 years ago

Yes.

lyn0102 commented 4 years ago

Thank you for your work, could you share the configuration of aotoslim training?

HaloTrouvaille commented 4 years ago

thank you for your reply, in my opinion ,i can use your us config to train a model and use slimming to find a best model(after prunning)? by the way ,i see not getattr(m, 'linked', False) in slimming function，it's mean residual_connection in us_mobilenet_v2.py?

Hello, can I add your wechat to ask some question? Thank you!