Closed LSXAxeller closed 3 weeks ago
Sorry,llama.cpp does not support custom chat_template. You can try using the package named "llama-cpp-python", which supports custom chat_template(example:[
reference:https://huggingface.co/IndexTeam/Index-1.9B-Chat-GGUF
Sorry,llama.cpp does not support custom chat_template. You can try using the package named "llama-cpp-python", which supports custom chat_template(example:[]sytem_message[reserved_0]user_message[reserved_1]response).
reference:https://huggingface.co/IndexTeam/Index-1.9B-Chat-GGUF
I am already using the sliced prompt as mentioned above, but it still have the spaces between words chars, and my frontend isn't based on python so I can't refer to llama-cpp-python, still this shouldn't be necessary since it's prompt issue not inference one, it depends on the dataset format not the backend
and [<unk>]sytem_message[reserved_0]user_message[reserved_1]response
doesn't really give enough context, the repo should have a tested prompt format instead placeholders like sytem_message, user_message, response
Suggestions:
We will give some test prompt later!!!
I am using llama.cpp to load models, and the fork I use doesn't support apply custom chat_template and the prompt template on hugging face doesn't work correctly
[<unk>]sytem_message[reserved_0]user_message[reserved_1]response
like thatEven after removing sytem_message, user_message, response words and the new lines, it still responding with empty spaces between words chars And using Alpaca instruct prompt template and even no prompt template it gives same results
Sample
Model: Index-1.9B-Character-GGUF-Q6_K