Open lllAlexanderlll opened 1 month ago
Implementation of Best-fit Packing https://arxiv.org/pdf/2404.10830v2 TLDR: a scalable and efficient method that packs documents into training sequences through length-aware combinatorial optimization.
Inspiration:
More efficient training with cleaner data batches.
Feature request
Implementation of Best-fit Packing https://arxiv.org/pdf/2404.10830v2 TLDR: a scalable and efficient method that packs documents into training sequences through length-aware combinatorial optimization.
Inspiration:
Motivation
More efficient training with cleaner data batches.