Make field more robust with None/nan repeated input

mlcommons / croissant

Croissant is a high-level format for machine learning datasets that brings together four rich layers.

https://mlcommons.org/croissant

Apache License 2.0

452 stars 41 forks source link

Closed ccl-core closed 1 month ago

ccl-core commented 1 month ago

Changes in field (more modular, more robust with none/nan values).
Add test with csv with nan values.
Add https://huggingface.co/datasets/teknium/OpenHermes-2.5?row=5 as an example of a dataset with repeated Null values for e2e test.

github-actions[bot] commented 1 month ago

MLCommons CLA bot All contributors have signed the MLCommons CLA ✍️ ✅