search

LostRuins / koboldcpp

A simple one-file way to run various GGML and GGUF models with KoboldAI's UI
https://github.com/lostruins/koboldcpp
GNU Affero General Public License v3.0
4.35k stars 312 forks source link

Add Phi-3 Mini instruct tag preset to Lite #891

Closed ANVF-Postal closed 4 weeks ago

ANVF-Postal commented 4 weeks ago

Hello! This is my first time contributing to a public Git repo. Apologies if I'm not doing this right.

This PR simply adds an instruct tag preset for the Microsoft Phi-3 Mini model, accessible through the dropdown menu in Kobold Lite. The same preset will also work for most models in the Phi-3 family.

Phi-3 is a pretty high performance model for its size, and it can run well on lower end systems (4.46T/s on GTX 780 3GB!). A GGUF file is also readily available from Microsoft.

I've tested this PR, and it doesn't spit out garbage.

Kobold-Phi-PR

With the ever-growing selection of LLMs being made available to the general public, the current instruct tag preset system should probably be replaced with a more flexible preset menu, where users can save their own presets. A big, updated catalogue of instruct presets for various models would also make Kobold much easier to use, and people wouldn't need to track down the proper prompt templates for each model.

Unfortunately, that's far out of scope for my programming capabilities. I just thought I'd suggest it.

Thanks a lot for taking the time to check out my PR! I suppose if anyone wants to suggest tag presets to add, I can follow this up with another pull request.

LostRuins commented 4 weeks ago

Thanks for the contribution, although next time please direct all lite related stuff to the kobold lite repo here: https://github.com/LostRuins/lite.koboldai.net/

For now i'll manually transfer the changes over and merge them in.

LostRuins commented 4 weeks ago

Merged manually.

ANVF-Postal commented 4 weeks ago

Good to know. I'll keep that in mind for next time. Thanks, Lost!