liucongg / ChatGLM-Finetuning

基于ChatGLM-6B、ChatGLM2-6B、ChatGLM3-6B模型,进行下游具体任务微调,涉及Freeze、Lora、P-tuning、全参微调等
2.63k stars 292 forks source link

问题 #137

Open wuguangshuo opened 7 months ago

wuguangshuo commented 7 months ago
def __call__(self, batch):
    lengths = [len(instance["input_ids"]) for instance in batch]
    batch_max_len = max(lengths)

    input_ids_batch, labels_batch = [], []
    for instance in batch:
        input_ids = instance["input_ids"]
        labels = instance["labels"]

        padding_len = batch_max_len - len(input_ids)
        input_ids = input_ids + [self.pad_token_id] * padding_len
        labels = labels + [-100] * padding_len

        input_ids_batch.append(input_ids)
        labels_batch.append(labels)

   input_ids不应该在左面填充pad吗