search

henk717 / KoboldAI

KoboldAI is generative AI software optimized for fictional use, but capable of much more!
http://koboldai.com
GNU Affero General Public License v3.0
359 stars 130 forks source link

Merge exllama backend into united. #447

Closed pi6am closed 12 months ago

pi6am commented 1 year ago

Add a new inference model backend based on exllama. Most of the work on this backend was done by Occam. My main contribution was discovering and working around a bug in torch.multinomial, hooking up stoppers, configuring bad_words_ids, and some other minor bug fixes.