search

horseee / LLM-Pruner

[NeurIPS 2023] LLM-Pruner: On the Structural Pruning of Large Language Models. Support Llama-3/3.1, Llama-2, LLaMA, BLOOM, Vicuna, Baichuan, TinyLlama, etc.
https://arxiv.org/abs/2305.11627
Apache License 2.0
860 stars 100 forks source link

I encountered the following error message when I assign iterative_steps = 2 during baichuan-7B pruning #21

Open yangd85 opened 1 year ago

yangd85 commented 1 year ago

Traceback (most recent call last): File "/home/jovyan/honor/yangdong/LLM-Pruner-main/examples/baichuan.py", line 342, in main(args) File "/home/jovyan/honor/yangdong/LLM-Pruner-main/examples/baichuan.py", line 229, in main pruner.step() File "/home/jovyan/honor/yangdong/LLM-Pruner-main/LLMPruner/torch_pruning/pruner/algorithms/metapruner.py", line 186, in step for group in self.prune_local(): File "/home/jovyan/honor/yangdong/LLM-Pruner-main/LLMPruner/torch_pruning/pruner/algorithms/metapruner.py", line 245, in prune_local imp = self.estimate_importance(group, ch_groups=ch_groups, consecutive_groups=consecutive_groups) File "/home/jovyan/honor/yangdong/LLM-Pruner-main/LLMPruner/torch_pruning/pruner/algorithms/metapruner.py", line 190, in estimate_importance return self.importance(group, ch_groups=ch_groups, consecutive_groups=consecutive_groups) File "/opt/miniconda3/envs/flash/lib/python3.9/site-packages/torch/autograd/grad_mode.py", line 27, in decorate_context return func(*args, **kwargs) File "/home/jovyan/honor/yangdong/LLM-Pruner-main/LLMPruner/pruner/hf_baichuan_pruner.py", line 180, in call local_norm = local_norm[idxs] IndexError: index 3840 is out of bounds for dimension 0 with size 3840

horseee commented 1 year ago

Our code currently does not support iterative_steps > 1 for baichuan. Please try iterative_steps = 1.