search

henk717 / KoboldAI

KoboldAI is generative AI software optimized for fictional use, but capable of much more!
http://koboldai.com
GNU Affero General Public License v3.0
359 stars 130 forks source link

gptq is soooo lazy #424

Closed one-some closed 1 year ago

one-some commented 1 year ago

now to figure out what makes them load so fast

note: quant_v1 is untested because i couldnt find a quant_v1 model, v2 and v3 are both tested