Open theblackcat102 opened 1 year ago
An easy fix would be setting padding_side = 'left' in DialogueDataCollator __post_init__ function
@dataclass
class DialogueDataCollator:
...
def __post_init__(self):
assert self.tokenizer.eos_token
self.tokenizer.padding_side = 'right'
Since llama models are using left padding, the supervised training dialoguecollator would cause the label_mask to pad in a different direction as the tokenizer.pad (input_ids, attention_mask), as torch.stack (label_mask) implements the right padding strategy.
Printing out the dataloader results in trainer_sft.py would also verify the issue
I think there's no padding_side assigned to right in the trainer_sft.py pipeline, so by default llama models we have trained are bit faulty