search

NVIDIA / Megatron-LM

Ongoing research training transformer models at scale
https://docs.nvidia.com/megatron-core/developer-guide/latest/user-guide/index.html#quick-start
Other
10.13k stars 2.28k forks source link

added the dataset packing #801

Closed shamanez closed 5 months ago

shamanez commented 5 months ago

Dataset packing was not avialable in the pre processing code. So it is better to add it.