huggingface / chug

Minimal sharded dataset loaders, decoders, and utils for multi-modal document, image, and text datasets.
Apache License 2.0
151 stars 10 forks source link

Bam! #3

Closed rwightman closed 6 months ago

rwightman commented 6 months ago

Kind of a lot of changes...