A Few issues with reproducing the code

[ctsan]

Hello,

I am trying to run your codebase. I am having some issues however:

1. Set of python requirements cannot be installed due to incompatibilities. Are these requirements strict or can they be relaxed?

2. After relaxing the above evaluation runs fine, but training requires a --distillation_path. Could you provide an example on how to use this argument?

3. To overcome 2, I set variable additional_args.do_distill to False. this results in an epoch being trained but crashing at the end. Model loss succesfully reduces but reg loss and lag loss is 0. The error at the end is a failure in assertion: " assert "head" in self.types" in the l0 module

Could you help me or provide pointers on resolving the above?

Thank you

[xiamengzhou]

Hi,

Thanks for using our repository!

1. I believe the incompatibility issue is from pytorch installation. Please refer to https://pytorch.org/get-started/previous-versions/ to install 1.10.2 version. You can use pip to install the rest of the datasets.
2. A path that contains a finetuned model pytorch_model.bin should work. You can use our script to get a finetuned model by setting PRUNING_TYPE=None to fall back to standard finetuning.
3. I will check it and get back to you soon.

[ctsan]

Hi,

Thank you for the response. I am still having some issues with 2. To my understanding, your current example for finetuning is to finetune a pruned model, but I don't have the pruned model either. Does your script support finetuning the unpruned model? and if so what are the arguments?

EDIT: I resolved 2, but I am still getting an error at 3.

[ctsan]

Found the error for 3. Example script had a typo

[xiamengzhou]

Hi,

Sorry for getting back to you! What was the typo you found from the example script?

[ctsan]

PRUNING_TYPE=structured_head+structured_mlp+hidden+layer

-> structured_heads is expected in the code I believe, not structured_head, or something along these lines.

[xiamengzhou]

Thanks! I just fixed it.