search

allenai / dolma

Data and tools for generating and inspecting OLMo pre-training data.
https://allenai.github.io/dolma/
Apache License 2.0
894 stars 90 forks source link

Better Filters Error Handling #171

Closed soldni closed 3 weeks ago

soldni commented 2 months ago

This PR refactors the jaq-based query selector (jq mode) so it raises appropriate error if non-existent key is requested. jsonpath_rs selector is left as-is since it never returns errors when keys don't exist.