guinmoon / LLMFarm

llama and other large language models on iOS and MacOS offline using GGML library.
https://llmfarm.site
MIT License
1.05k stars 62 forks source link

Prompt format for SeaLLM model? #64

Closed duyluandethuong closed 3 weeks ago

duyluandethuong commented 1 month ago

I'm trying to use this model: https://huggingface.co/SeaLLMs/SeaLLM-7B-v2.5-GGUF

The SeaLLM-7B-v2.5 has the following preset for LM studio: https://huggingface.co/SeaLLMs/SeaLLM-7B-v2.5-GGUF/blob/main/seallm-v2.5.preset.json

I don't know how to convert this into prompt format for LLMFarm. Please help.

When I try to run in Xcode, in print the following lines

Screenshot 2024-05-19 at 10 52 14 AM
guinmoon commented 1 month ago

Try this: BOS = false, EOS = false

[system](<|im_start|>system
You are a helpful assistant.)
<eos><|im_start|>user
{prompt}<eos><|im_start|>assistant

stop words = <|im_start|>,<eos>

guinmoon commented 1 month ago

Tell me this template helped you?