feat: User can use Stop Words presets and add custom

[dan-homebrew]

Problem

• There are many users looking to enter multiple stop words
• The "stop" section for configuring the model does not make it clear whether it should be a comma-separated list, a newline-separated list, or something else.
• It would be nifty (and I think aligned with how other apps do it) if as you insert a token, it creates a standalone entry in the box. - Something like: <endofstring>, <new_sentence>, <this_is_the_end>.
• And then you could individually click-to-delete any of the "stop" tags you created. Hopefully that is comprehensible.

Solution

From technical aspect, there are 2 cases:

• Models from cortexso source: the stop token is included in .yaml file, so Jan app just read from it to place it as default.
• Models from other source: With cortex.cpp, when read model from gguf file, it also contains the stop words list, cortex.cpp will read and write to .yaml, Jan app can read from .yaml to set as default

Format

Most of model introduce stop tokens with this format <{content}> . The content is different for each model arch. So I think we can predefine a list option of map model's arch : list stop words like this for user to choses:

• llama3: ["<|eot_id|>", "<|end_of_text|>"]
• mistral: ["< /s > "]
• ...

Maybe 5 or 6 popular models arch is enough and another option to let users input whatever they want (this feature may be only for power user or dev because normal user might only use default configuration)

Design

Figma: https://www.figma.com/design/DYfpMhf8qiSReKvYooBgDV/Jan-App-(3rd-version)?node-id=8281-97234&t=qb7yU8r2PAayVdNW-4

• Use a tag-like interface where each stop word is in its own removable "pill"
• Users can add new tags and remove existing ones

Default Stop Words:

• These come from the model's YAML file
• Should be visually distinct and are recommended not to be removed (they're usually carefully chosen by the model creators for optimal performance, removing these could potentially cause issues with the model's behavior or output quality).

• Users should understand these are recommended for the model

  

Predefined Options:

Offer a dropdown preset or quick-select for common stop words based on model architecture

Custom/User-Added Stop Words:

• Added by the user & should be removable

---

Task

• [x] Figure out what stop tokens format should be in @nguyenhoangthuan99
• [x] Design @imtuyethan
• [ ] UI implementation @urmauur

[dan-homebrew]

@nguyenhoangthuan99 however there are some clarifications needed from Inference team. What is the stop token format?

• I think the existing "Stop word" (as per the UX above) is incorrect
• We need to be technically accurate
• Should we prefill the <> for the user?

<|special_token|>
<|end_of_text|>
<|eom_id|>

[nguyenhoangthuan99]

Stop words of a model can be a list so I think we can make it like this

From technical aspect, I think there are 2 cases we can follow:

• Models from cortexso source: the stop token is included in <model_id>.yaml file, so Jan app just read from it to place it as default.
• Models from other source: With cortex.cpp, when read model from gguf file, it also contains the stop words list, cortex.cpp will read and write to <model_id>.yaml, Jan app can read from <model_id>.yaml to set as default

Format Most of model introduce stop tokens with this format <{content}> . The content is different for each model arch. So I think we can predefine a list option of map model's arch : list stop words like this for user to choses:

• llama3: ["<|eot_id|>", "<|end_of_text|>"]
• mistral: ["< /s > "] ...

Maybe 5 or 6 popular models arch is enough and another option to let users input whatever they want (this feature may be only for power user or dev because normal user might only use default configuration)