allenai / dolma

Data and tools for generating and inspecting OLMo pre-training data.
https://allenai.github.io/dolma/
Apache License 2.0
972 stars 107 forks source link

'File partition' option and 'document' directory specification #213

Closed Whattabatt closed 4 weeks ago

Whattabatt commented 1 month ago

Allows for partitioning work at the file level - checking the hash of the de-globbed file names instead of spans. Also, creates an optional 'document dir' parameter for dealing with source data that doesn't have a 'document' directory to replace with 'attributes' (previously the attribute path generator would try replacing 'documents', not find any, and use the source directory!)