Experimental results - Githubissues

princeton-nlp / CoFiPruning

[ACL 2022] Structured Pruning Learns Compact and Accurate Models https://arxiv.org/abs/2204.00408

MIT License

188 stars 32 forks source link

Experimental results #29

Closed iMountTai closed 1 year ago

iMountTai commented 1 year ago

Hello,@xiamengzhou !The result on the Squad task dataset is 79.74, which is quite different from the result (82.6) in the paper. Can you further announce the detailed parameters? The teacher model F1 value is 88.43. I will be very grateful!

xiamengzhou commented 1 year ago

Hi,

A few details are worth looking at for SQuAD:

We reported results of a 92.4% sparsity for SQuAD, as shown in README.md, pruning it to 95% would lead to a 1-2 points drop.
Make sure that you finetune the model after finalizing the structure! It makes a huge difference for highly pruned models.

Let me know if you have more questions!

iMountTai commented 1 year ago

Thanks!