search

bigscience-workshop / t-zero

Reproduce results and replicate training fo T0 (Multitask Prompted Training Enables Zero-Shot Task Generalization)
Apache License 2.0
456 stars 53 forks source link

Fix padding side #32

Closed thomasw21 closed 2 years ago

thomasw21 commented 2 years ago

Fixes: #29

As @kkawamu1 noticed, <pad> between inputs and targets is bad. Right now I've chose to use padding_side="left" as a fix. The current fix does:

<INPUT> <INPUT_PAD> <TARGET> <TARGET_PAD> -> <INPUT_PAD> <INPUT> <TARGET> <TARGET_PAD>

The more permanent fix is to to do:

<INPUT> <INPUT_PAD> <TARGET> <TARGET_PAD> -> <INPUT> <TARGET> <INPUT_PAD> <TARGET_PAD>

This becomes especially useful when you start to leave the text input | text target paradigm so multiple step one.

VictorSanh commented 2 years ago

good to go! feel free to merge @thomasw21 !