search

NVIDIA / NeMo-Curator

Scalable data pre processing and curation toolkit for LLMs
Apache License 2.0
482 stars 58 forks source link

Add workflow for running cpu pytests #13

Closed ayushdg closed 6 months ago

ayushdg commented 6 months ago

@ryantwolf Looks like Nemo-toolkit install fails in a python 3.9 environment (can reproduce locally). I can skip 3.9 tests here for the time being, but not sure if it's supposed to work with 3.9 or not.

ryantwolf commented 6 months ago

@ayushdg hm, why does it fail with 3.9? I can ask around if it's supposed to work. I see the logs now. We can skip for now though.

After talking with folks, looks like NeMo only supports Python 3.10 and above now. So we should bump our minimum version too.

ayushdg commented 6 months ago

After talking with folks, looks like NeMo only supports Python 3.10 and above now. So we should bump our minimum version too.

Thanks, I'll limit the scope of the PR to 3.10 testing. We can bump the minimum version in a followup.