search

NolanoOrg / sparse_quant_llms

SparseGPT + GPTQ Compression of LLMs like LLaMa, OPT, Pythia
41 stars 3 forks source link

sparseGPT tool #1

Open YixinSong-e opened 1 year ago

YixinSong-e commented 1 year ago

When will the tool be ready for use?

Ayushk4 commented 1 year ago

I don't have an exact date which I can offer you.

As of now quantization with sparsification is not supported I would appreciate if you could run it and compare results with the paper.

YixinSong-e commented 1 year ago

Thanks for your reply, I am very interested in your work and I am looking forward to the completion of the sparsification + quantization function. And I will try sparsification! :)

YixinSong-e commented 1 year ago

Can I test the model of rwkv in repo now?

Ayushk4 commented 1 year ago

OPT is nearly completed - I just haven't tested it against results.

RWKV is not ready

Ayushk4 commented 1 year ago

https://github.com/IST-DASLab/sparsegpt The authors have released their codebase.