[FEA] Add support for Multiple Model Quality Classification

[sarahyurick]

In previous versions of NeMo Curator, we supported multiple model quality classification with a combination of Slurm and Python scripts. These scripts were designed to allow the user to pass in multiple model paths at once for running multi-node multi-GPU data classification.

Now, we are moving away from Slurm scripts in favor of a Python API. I think we should eventually create a Python API (ideally using Crossfit) to support multiple model classification.

[sarahyurick]

For reference:

• https://github.com/sarahyurick/mnmg_classifier_inference/blob/main/examples/distributed_data_classification_examples/run_multiple_quality_models.sh
• https://github.com/sarahyurick/mnmg_classifier_inference/blob/main/ndc/distributed_data_classification/quality_classifier_multiple_models_inference.py