search

huggingface / alignment-handbook

Robust recipes to align language models with human and AI preferences
https://huggingface.co/HuggingFaceH4
Apache License 2.0
4.2k stars 357 forks source link

`apply_chat_template` not compatible with tokenizers that do not support a system prompt #101

Closed nathan-az closed 5 months ago

nathan-az commented 6 months ago

Noticed this while trying to run SFT on mistralai/Mistral-7B-Instruct-v0.2. It seems some models simply don't support a system prompt. Error was the following:

TemplateError: Conversation roles must alternate user/assistant/user/assistant/....

My dataset does not include a system prompt. The behaviour is likely due to the empty system prompt being injected in apply_chat_template.

Happy to make a PR to change this behaviour and not inject an empty system prompt, but not sure if we want to. The reverse may be true for some tokenizers, where a system prompt is absolutely required. Maybe a new argument inject_system_prompt or requires_system_prompt? There may also be a way to inspect the jinja templates to determine, but I haven't looked closely at this.

nathan-az commented 6 months ago

I think this should be as simple as checking the tokenizer's template (or default_chat_template) and seeing if system appears in it, before injecting the empty system message.

It sounds simple but I can't think of any cases in which it would fail. Can make a PR in the next few days.

Feynman27 commented 6 months ago

Same issue as https://github.com/huggingface/alignment-handbook/issues/93

nathan-az commented 5 months ago

Resolved by above