Open source the pruning mask.

princeton-nlp / LLM-Shearing

[ICLR 2024] Sheared LLaMA: Accelerating Language Model Pre-training via Structured Pruning

https://arxiv.org/abs/2310.06694

MIT License

533 stars 39 forks source link

Closed Achazwl closed 3 months ago

Achazwl commented 4 months ago

Can you please upload your pruning mask related to Sheared-LLaMA-1.3B-Pruned and Sheared-LLaMA-2.7B-Pruned?

This would help researchers conduct further analysis on weight importance, sparsity, etc.

xiamengzhou commented 3 months ago

Hi @Achazwl sorry for the late reply!

We uploaded masks here, hope it helps!

Achazwl commented 3 months ago

Thanks a lot!