About the upper layer in your paper

princeton-nlp / CoFiPruning

[ACL 2022] Structured Pruning Learns Compact and Accurate Models https://arxiv.org/abs/2204.00408

MIT License

189 stars 31 forks source link

Closed CaffreyR closed 2 years ago

CaffreyR commented 2 years ago

Hi @xiamengzhou , many thanks to your contribution. I have a small question in your paper, in your paper you said that

CoFi tends to prune submodules more from upper layers than lower layers.

What is the upper layer means? Is it near input or output? Many thanks!

xiamengzhou commented 2 years ago

Upper layers mean the layers that are closer to the output!

CaffreyR commented 2 years ago

Many thanks! :)