search

Eric-mingjie / rethinking-network-pruning

Rethinking the Value of Network Pruning (Pytorch) (ICLR 2019)
MIT License
1.51k stars 293 forks source link

when should I train it？ #46

Closed hlyj1995 closed 3 years ago

hlyj1995 commented 4 years ago

Hi，I want to pure my model on tracking task. I‘m using resnet22 and my tracking datasets, train it by my program.So I have a trouble ,when should I train it? If I want to achieve a 60% pruning rate, Then I should first prune the randomly weighted model to 60% and then train it, or gradually prune the model to do a fine tuning and pruning process. look forward your help

Eric-mingjie commented 4 years ago

Prune the model you trained and then finetune.

hlyj1995 commented 4 years ago

@Eric-mingjie Thanks for your reply. If I prune my trained model and finetune it . Isn't this a violation of your paper's idea of retraining using a model with random initial weights? And I have tried your code, so there is a question, I found some pruning needs to download the pre-training model, are these models randomly weighted?

Eric-mingjie commented 4 years ago

This isn't a violation. Our paper is just saying that training from scratch achieves similar performance as fine-tuning. In practice, fine-tuning is much faster if you have a pre-trained model.

Pre-trained models are not randomly weighted.

hlyj1995 commented 4 years ago

This isn't a violation. Our paper is just saying that training from scratch achieves similar performance as fine-tuning. In practice, fine-tuning is much faster if you have a pre-trained model.

Pre-trained models are not randomly weighted.

thanks,i get it