allenai / dolma

Data and tools for generating and inspecting OLMo pre-training data.
https://allenai.github.io/dolma/
Apache License 2.0
894 stars 90 forks source link

Added Support for JQ syntax in include/exclude mixer config #131

Closed soldni closed 6 months ago