mosaicml / examples

Fast and flexible reference benchmarks
Apache License 2.0
435 stars 124 forks source link

Seq2Seq finetuning #304

Closed alextrott16 closed 1 year ago

alextrott16 commented 1 year ago

This PR covers seq-to-seq finetuning (such as IFT) for LLMs. These changes create a new dataloading codepath. All major changes are within llm/src/data/finetuning/.

At a high level:

Other changes:

alextrott16 commented 1 year ago

Note that this will need to be partially re-written after https://github.com/mosaicml/examples/pull/314 goes through.