search

Lightning-Universe / lightning-transformers

Flexible components pairing 🤗 Transformers with :zap: Pytorch Lightning
https://lightning-transformers.readthedocs.io
Apache License 2.0
607 stars 77 forks source link

Shuffle support #272

Closed mahnerak closed 1 year ago

mahnerak commented 1 year ago

After looking the codebase I've discovered there's no shuffling while training. Is there any reason for it?

https://github.com/Lightning-AI/lightning-transformers/blob/f439c954fd085499ddde4ed27f927ba70d079595/lightning_transformers/core/data.py#L174-L181

Borda commented 1 year ago

@SeanNaren, mind checking this out? :otter:

Borda commented 1 year ago

closing in favor of #298