Closed ghost closed 1 year ago
get_masks_and_position_ids函数中 attention_mask[..., : mask_position - 1] = 1 是不是要改成attention_mask[..., : seq_len - 1] = 1 ? 参考官方代码def get_attention_mask(tokenizer, input_ids, device): attention_mask[..., :context_len] = 1,此处的context_len = seq_len - 1
get_masks_and_position_ids函数中 attention_mask[..., : mask_position - 1] = 1 是不是要改成attention_mask[..., : seq_len - 1] = 1 ? 参考官方代码def get_attention_mask(tokenizer, input_ids, device): attention_mask[..., :context_len] = 1,此处的context_len = seq_len - 1