Closed chtanch closed 6 months ago
Detection and auto-loading of Instruct template for Qwen-7b added. It is pointless to do the same for ChatGLM3. This is because ` <ChatGLM3 model folder>/tokenizer_config.json
contains a template, which is placed at a higher priority than custom Instruct templates by webUI. However, the template in tokenizer_config.json seems to be incorrect and does not work well.
Usage:
References used for template format:
Qwen - https://github.com/vllm-project/vllm/issues/1914 ChatGLM3 - https://github.com/THUDM/ChatGLM3/blob/main/PROMPT_en.md
Generated input with formatting tokens
Qwen:
chatGLM3:
Results
Results for Qwen-7b seem to be better than with the hardcoded function. Results for ChatGLM3-6b seem to be better than with default chat template.