search

CASIA-IVA-Lab / FLAP

[AAAI 2024] Fluctuation-based Adaptive Structured Pruning for Large Language Models
https://arxiv.org/abs/2312.11983
Apache License 2.0
35 stars 10 forks source link

Compute sparsity problem #10

Open ironartisan opened 5 months ago

ironartisan commented 5 months ago

check_sparsity return Ratio of the count of non-zero weights to total parameters in the model. How is non-zero sparsity calculated? https://github.com/CASIA-IVA-Lab/FLAP/blob/3bb57db3449dd2fa04a5c2192de80e87e33be2b1/lib/prune.py#L43