search

huggingface / nn_pruning

Prune a model while finetuning or training.
Apache License 2.0
394 stars 58 forks source link

Dependencies and version numbers? #4

Closed lewtun closed 3 years ago

lewtun commented 3 years ago

Hello François,

First of all, thank you for making your work on movement-pruning and sparse transformers available to the public!

I've finally found some time to start playing around with nn_pruning and I noticed there appears to be a mismatch the dependencies listed in setup.py (i.e. just click) vs those visible in the source code (e.g. transformers, datasets, torch etc).

Would you mind sharing which package dependencies and version numbers you used to run your experiments?

madlag commented 3 years ago

Yes, I have a lot of cleanup to do in the code right now, as it was based initially on the movement pruning code , and some part of the repository is obsolete. You can just use the latest versions of transformers / datasets and torch, there should be no problem, and of course if there is any tell me.

lewtun commented 3 years ago

Thanks for the information!

I think one place where the code needs updating to be compatible with transformers v4.3.3 is the SparseTrainer.compute_loss(model, inputs) function to match the new signature of Trainer.compute_loss(model, inputs, return_outputs)

I'd be happy to help out once I've become a bit more familiar with the nn_pruning codebase 😄

madlag commented 3 years ago

OK, I am going to update transformers and fix this ! Don't hesitate to ping me on Slack so we can have a live working session to speed things up.