-
Hi and thanks for the amazing repo.
I have a bit of tall request. SparseGPT uses a per-layer optimal brain surgeon approach to pruning. Here is the [pytorch code](https://github.com/IST-DASLab/spar…
-
Sparse should reduce the size and increase infer speed without hurting perf too much. This repo https://github.com/IST-DASLab/sparsegpt is Apache license and may be useful (I hope)
-
When will the tool be ready for use?
-
-
-
We have an existing model [neuralmagic/SparseLLama-2-7b-ultrachat_200k-pruned_50.2of4](https://huggingface.co/neuralmagic/SparseLLama-2-7b-ultrachat_200k-pruned_50.2of4) that has already been pruned t…
mgoin updated
3 weeks ago
-
Hello, I find it extremely slow to do sparsegpt with tp=1 and pp=1. Will a larger number help? Thank you!
-
```
(textgen) [root@pve-m7330 sparsegpt]# python llama.py ../text-generation-webui/models/TinyLlama-1.1B-Chat-v1.0/ wikitext2 --nsamples 10
Token indices sequence length is longer than the specified…
-
**Describe the bug**
A clear and concise description of what the bug is.
**Hardware details**
Information about CPU and GPU, such as RAM, number, etc.
**Software version**
Version of relevant…
-
Your project are really helpful!
I am also trying to reproduce SparseGPT:
https://arxiv.org/pdf/2301.00774.pdf
Does the SparseGPT you tried work?