How to select parts to bp in sft

huggingface / alignment-handbook

Robust recipes to align language models with human and AI preferences

https://huggingface.co/HuggingFaceH4

Apache License 2.0

4.18k stars 354 forks source link

How to select parts to bp in sft #138

Open Fu-Dayuan opened 3 months ago

Fu-Dayuan commented 3 months ago

As the pic has shown, there are some cases that some parts of the gpt's response should not be cacluated in backward computing, if I want to achieve this function, what should I do? (or can you realize this in a new version?)