Closed yafuly closed 7 months ago
Hi,
Thank you for your excellent work and code implementation.
I observed that you manually shifted the labels in the data collator. However, it appears that Huggingface automatically handles label shifting within the model class, as demonstrated in [Huggingface source code] (https://github.com/huggingface/transformers/blob/0ea42ef0f9f71deba7775ead33afa0e493823d60/src/transformers/models/llama/modeling_llama.py#L1081C5-L1081C5).
Hi,
Thank you for your excellent work and code implementation.
I observed that you manually shifted the labels in the data collator. However, it appears that Huggingface automatically handles label shifting within the model class, as demonstrated in [Huggingface source code] (https://github.com/huggingface/transformers/blob/0ea42ef0f9f71deba7775ead33afa0e493823d60/src/transformers/models/llama/modeling_llama.py#L1081C5-L1081C5).