Adding _warmup_mmap_file function missing from MMapIndexedDataset

EleutherAI / pythia

The hub for EleutherAI's work on interpretability and learning dynamics

Apache License 2.0

2.16k stars 156 forks source link

Closed rdiehlmartinez closed 5 months ago

rdiehlmartinez commented 5 months ago

_warmup_mmap_file function copied over from fairseq. Seems to be missing from mmap_dataset

CLAassistant commented 5 months ago

All committers have signed the CLA.