Can prune model convert to llama.cpp ggml？

horseee / LLM-Pruner

[NeurIPS 2023] LLM-Pruner: On the Structural Pruning of Large Language Models. Support Llama-3/3.1, Llama-2, LLaMA, BLOOM, Vicuna, Baichuan, TinyLlama, etc.

https://arxiv.org/abs/2305.11627

Apache License 2.0

834 stars 98 forks source link

Can prune model convert to llama.cpp ggml？ #16

Open shaonianyr opened 1 year ago

horseee commented 1 year ago

Hi, we have not tried to convert the pruned model to llama.cpp.