search

laelhalawani / glai

glai - GGUF LLAMA AI - Package for simplified model handling and text generation with Llama models quantized to GGUF format. APIs for downloading and loading models automatically, includes a db with models of various scale and quantizations. With this high level API you need one line to load the model and one to generate text completions.
https://laelhalawani.github.io/glai/
Other
5 stars 0 forks source link

Include support for system message in AI messages #30

Closed laelhalawani closed 6 months ago

laelhalawani commented 6 months ago

Some Llama2 models are fine-tuned with system prompt (i.e. phi2) to utilize them properly system prompt support is needed.

laelhalawani commented 6 months ago

This was added in latest updates version 0.0.15 full support to for system prompts has been added to both messages, model db and model data, now if you're loading a model that supports system prompts you can provide system prompt by using aai.generate(user_message="Your prompt", system_prompt="You are a helpful assistant.") To check if the model supports system prompts you can use aai.model_data.has_system_tags().