search

huggingface / nn_pruning

Prune a model while finetuning or training.
Apache License 2.0
393 stars 57 forks source link

Experiments run slowly on V100 #28

Closed xiamengzhou closed 2 years ago

xiamengzhou commented 2 years ago

Hi thanks for sharing this great work! I am trying to train a model on my own with the hybrid filled setting for MNLI and it seems that it takes more than 5 hours to train for one epoch. Is the training expected to be this slow or there might be something run with what I do? I basically use the same hyperparameters provided in the analysis folder.

xiamengzhou commented 2 years ago

I accidentally set the nrows to be 1, by setting it to 32 it's much faster.

iMountTai commented 1 year ago

@xiamengzhou Does this script run with the hybrid-filled method by default?Thanks!