search

horseee / LLM-Pruner

[NeurIPS 2023] LLM-Pruner: On the Structural Pruning of Large Language Models. Support Llama-3/3.1, Llama-2, LLaMA, BLOOM, Vicuna, Baichuan, TinyLlama, etc.
https://arxiv.org/abs/2305.11627
Apache License 2.0
880 stars 106 forks source link

Question related to the model tuning #39

Open shawnricecake opened 12 months ago

shawnricecake commented 12 months ago

Hi,

Great work first!

I am confused with the model tuning part.

According to the code, it seemed that you used the lora method. This, in my opinion, will destroy the sparsity you have made in the original model after merging the lora weights to the model weights.

could you explain this?

Thanks Shawn

VainF commented 12 months ago

Hi @shawnricecake, LLM-Pruner is a structural method and thus produces a dense model after pruning.

shawnricecake commented 11 months ago

Hi @shawnricecake, LLM-Pruner is a structural method and thus produces a dense model after pruning.

Hi, thanks for your reply, so, the model weights after merge the lora weights will be dense?

the main contribution of paper is the structure pruning?

Thanks Shawn