pytorch / text

Models, data loaders and abstractions for language processing, powered by PyTorch
https://pytorch.org/text
BSD 3-Clause "New" or "Revised" License
3.49k stars 815 forks source link

Add shift-right method to T5 model #2131

Closed yohann-benchetrit closed 1 year ago

yohann-benchetrit commented 1 year ago

We add a _shift_right method to the T5 Model. This is used for training (see Issue 2127).

It shifts an input batch of sequences of token-indices, one index to the right. Their first token is replaced with the start-of-sequence token, which is the padding-token index for T5 models

github-advanced-security[bot] commented 1 year ago

You have successfully added a new CodeQL configuration .github/workflows/codeql.yml:build. As part of the setup process, we have scanned this repository and found 3 existing alerts. Please check the repository Security tab to see all alerts.

joecummings commented 1 year ago

@yohann-benchetrit Can you add a unittest for this?

joecummings commented 1 year ago

@yohann-benchetrit Looks like there's an error in how the config is being called within _shift_right