search

serge-chat / serge

A web interface for chatting with Alpaca through llama.cpp. Fully dockerized, with an easy to use API.
https://serge.chat
Apache License 2.0
5.68k stars 406 forks source link

🚀 [Feature]: Add support for Prompt Templates #223

Open ghost opened 1 year ago

ghost commented 1 year ago

I'm pretty sure the Open-Assistant models added in 17308a8 require different labels than the normal llama models. I couldn't find anything on their llama huggingface repo, but in https://huggingface.co/OpenAssistant/stablelm-7b-sft-v7-epoch-3 I found this:

Prompting

Two special tokens are used to mark the beginning of user and assistant turns: <|prompter|> and <|assistant|>. Each turn ends with a <|endoftext|> token.

And I strongly believe this is the case for the llama version as well, because when I played around with it, it sometimes outputted those strings.

gaby commented 1 year ago

@pabl-o-ce Is this the stuff you mentioned on Discord?

pabl-o-ce commented 1 year ago

yes :)

k0gen commented 1 year ago

If you're referring to renaming chats with a custom name, that would be a must-have feature.