search

NVIDIA / NeMo-Curator

Scalable data pre processing and curation toolkit for LLMs
Apache License 2.0
611 stars 83 forks source link

Remove flaky PyTest #235

Closed sarahyurick closed 2 months ago

sarahyurick commented 2 months ago

Removes changes from https://github.com/NVIDIA/NeMo-Curator/pull/218. Since test_uneven_common_crawl_range is very flaky, we skip it for now. I will also open an issue for possibly re-adding it in the future.