How to convert to 4bit gptq

Stability-AI / StableLM

StableLM: Stability AI Language Models

Apache License 2.0

15.85k stars 1.04k forks source link

How to convert to 4bit gptq #44

Closed c-seeger closed 1 year ago

c-seeger commented 1 year ago

I've seen there is a 4bit gptq version of stablelm and i'm curious if someone could lead me to some resources describing how to convert the current AI model to 4bit gptq. Any hint would be much appreciated.