WoosukKwon / retraining-free-pruning

[NeurIPS 2022] A Fast Post-Training Pruning Framework for Transformers
https://arxiv.org/abs/2204.09656
173 stars 27 forks source link

Good Job! Can this framework work for the Transformer-based vision models? #6

Closed ybai62868 closed 1 year ago

WoosukKwon commented 1 year ago

Great question. The underlying principles in our work do not leverage any knowledge of the domain, and thus should be applicable to vision and audio transformers as well. That being said, we haven't got the chance to test whether it actually works. Please feel free to try it out and let us know your experience!