Fix text_chosen and text_rejected

huggingface / alignment-handbook

Robust recipes to align language models with human and AI preferences

https://huggingface.co/HuggingFaceH4

Apache License 2.0

4.53k stars 393 forks source link

Fix text_chosen and text_rejected #128

Closed chujiezheng closed 6 months ago

chujiezheng commented 7 months ago

We do not directly create text_chosen from chosen_messages (same for rejected), as chosen_messages contains only an assistant message and it may raise jinja error (raise_exception('Conversation roles must alternate user/assistant/user/assistant/...')) in many chat models' chat_template (example. Instead, we remove the prompt part from the whole chosen_messages's input text.

HuggingFaceDocBuilderDev commented 6 months ago

The docs for this PR live here. All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.

chujiezheng commented 6 months ago

@lewtun Hi, could you please check what's wrong with my commits/pr?

peter-peng-w commented 4 months ago

May I ask why this PR isn't reflected on the main branch?