KoboldAI is generative AI software optimized for fictional use, but capable of much more!
GNU Affero General Public License v3.0
352
stars
130
forks
source link
Exllamav2: Track norm, linear, and attn layer scratch space #515
Closed
one-some closed 3 months ago
Tested with Michel-13B-exl2 and LLaMA2-13B-Tiefighter-GPTQ