Share new re-quantized model

[Curlypla]

Since the new GPTQ-for-LLaMa commits it is necessary to re-quantize the models to be compatible. Can someone upload them (because the ones from decapoda-research are too old and do not work)

[s4rduk4r]

Also you can try torrents from here - https://github.com/oobabooga/text-generation-webui/pull/530#issuecomment-1483941105

Also my version of 13B - https://huggingface.co/sardukar/llama13b-4bit-v2

[Ph0rk0z]

Except those won't work with this patch. At least not yet.

[wassname]

These work for me https://huggingface.co/maderix/llama-65b-4bit/tree/main

[s4rduk4r]

These work for me https://huggingface.co/maderix/llama-65b-4bit/tree/main

These are GPTQv1 models. But they're work fine

[ehartford]

Also you can try torrents from here - oobabooga/text-generation-webui#530 (comment)

Also my version of 13B - https://huggingface.co/sardukar/llama13b-4bit-v2

is there a 30B version of this?

[s4rduk4r]

is there a 30B version of this?

Last time I've checked the torrent it contained 30B and 65B versions. You should try it

[ehartford]

I found it (I had already downloaded it and forgot) thank you

[s4rduk4r]

Here someone has uploaded all fresh 4bit models - https://huggingface.co/Neko-Institute-of-Science

[ehartford]

Thank you for the link! Hope it stays up