Open Curiosity007 opened 1 year ago
As this is yet to support GPTQ models, maybe you can extend the functionality by adding api support of textgen webui? Textgen Webui can load GPTQ model and can expose API
Infinitely seconding
sdrakulich
commented
1 year ago
As this is yet to support GPTQ models, maybe you can extend the functionality by adding api support of textgen webui? Textgen Webui can load GPTQ model and can expose API