search

huggingface / optimum-habana

Easy and lightning fast training of 🤗 Transformers on Habana Gaudi processor (HPU)
Apache License 2.0
153 stars 202 forks source link

fix lora padding loss problem #1503

Open ranzhejiang opened 2 days ago

ranzhejiang commented 2 days ago

We must set its label to IGNORE_INDEX for padding token, otherwise the loss can't align with unpadded way