search

IST-DASLab / ACDC

Code for reproducing "AC/DC: Alternating Compressed/DeCompressed Training of Deep Neural Networks" (NeurIPS 2021)
Apache License 2.0
20 stars 4 forks source link

Can I export the pruned model? #1

Closed detectRecog closed 2 years ago

detectRecog commented 2 years ago

Hello, great work! How to export the pruned model?

ohaijen commented 2 years ago

Hi detectRecog, thank you :) During training, the best sparse and dense checkpoints, as well as the last checkpoint, are saved automatically, it will be wherever you set your --experiments_root_path to, along with all other artifacts of the run, something like my_experiment_root_path/run_type/run_name/timestamp/best_sparse_checkpoint.ckpt or my_experiment_root_path/run_type/run_name/timestamp/last_checkpoint.ckpt. Please feel free to reopen if this doesn't seem to be happening for you.

detectRecog commented 2 years ago

@ohaijen Hi Jen, thank you for your rapid response! I find the location of the 'best_sparse_checkpoint.ckpt' file and I can observe the notably high sparsity achieved by ACDC. However, I don't know how to export the pruned network (such as using torch.jit.trace) with removing zero-weight channels and even conv blocks to accelerate the inference speed on the edge device. Do you have any ideas?

ohaijen commented 2 years ago

hi @detectRecog, if you are looking for real speedups at an edge device, you might prefer a pruning approach that gives structured pruning (ie, removes entire filters). Since AC/DC is generally unstructured, you actually keep most filters (95% sparsity on ResNet50 removes something like 20% of filters completely). Would you like me to point you to some structured pruning resources?

detectRecog commented 2 years ago

hi @detectRecog, if you are looking for real speedups at an edge device, you might prefer a pruning approach that gives structured pruning (ie, removes entire filters). Since AC/DC is generally unstructured, you actually keep most filters (95% sparsity on ResNet50 removes something like 20% of filters completely). Would you like me to point you to some structured pruning resources?

Yes, you're so kind. Thank you so much. I also notice that deepsparse starts to offer libraries for running sparse models on CPU. It is a good beginning for deploying sparse models. I believe sparse models are very promising on edge inference.

sophieyl820 commented 2 years ago

hi @detectRecog, if you are looking for real speedups at an edge device, you might prefer a pruning approach that gives structured pruning (ie, removes entire filters). Since AC/DC is generally unstructured, you actually keep most filters (95% sparsity on ResNet50 removes something like 20% of filters completely). Would you like me to point you to some structured pruning resources?

could you recommend some structured pruning methods?