search

huggingface / nn_pruning

Prune a model while finetuning or training.
Apache License 2.0
393 stars 57 forks source link

Does this script run with the hybrid-filled method by default?Thanks! #41

Open iMountTai opened 1 year ago

iMountTai commented 1 year ago

@madlag

sangamesh-kodge commented 1 year ago

I had the same question. I understand I need to set "attention_pruning_method" : pruning_method and "dense_pruning_method" : pruning_method:1d_alt for hybrid pruning.

Is it that for hybrid-filled approach we train the model again with reinitializing the weights for heads which are not completely pruned and this is done separately after hybrid pruning? or Is there a argument I can set in the code for hybrid-filled pruning? @madlag @echarlaix