What is the effect of the choice of padding strategy on transformer performance?

[kgreyy]

• DataLoader padding strategy
  • left vs right padding with attention masks
  • without attention masks
• Zero-padding

[zrkrlc]

By 'attention mask', you mean this, right? https://lukesalamone.github.io/posts/what-are-attention-masks/