Closed kft334 closed 1 year ago
I wonder if his changes can just be merged to GPTQ itself. That would make things easier. Some dataset py changes were only ones that alter GPTQ stuff. Unless the new commits massively change things, it's easier than maintaining a separate repo.
This issue has been closed due to inactivity for 30 days. If you believe it is still relevant, please leave a comment below.
Currently https://github.com/0cc4m/GPTQ-for-LLaMa/tree/gptneox has to be manually installed and a few lines have to be modified in webui to load the pre-quantized model. Would be good to automate this to be able to load it like LLaMa. I know that there will be better models but it's the best one that I've been able to run yet and it would make it more accessible.