Generating predictions with CoFi models

princeton-nlp / CoFiPruning

[ACL 2022] Structured Pruning Learns Compact and Accurate Models https://arxiv.org/abs/2204.00408

MIT License

188 stars 32 forks source link

Generating predictions with CoFi models #37

Closed eldarkurtic closed 1 year ago

eldarkurtic commented 1 year ago

Hi, First of all, thanks a lot for open-sourcing your code and models!

I've been trying to use your code to generate predictions with CoFi models (with --do_predict on for example test-split of GLUE tasks) but unfortunately the prediction loop always fails with CUDA OOM exception (even on the 80GB A100 GPU). Could you also please try and let me know if I did something wrong?

xiamengzhou commented 1 year ago

Hi, sorry for the late reply! Which script are you using exactly? Could you provide the command line?

eldarkurtic commented 1 year ago

Hey, no worries at all, it was a problem on my side. Sorry for spamming, and btw, really cool work! :-)