feat: support classification

[n0w0f]

There is lot of duplications, so as to avoid any breakage at this moment. might have to refactor later.

we could also choose to not merge it. but would be good to review the code.

Summary by Sourcery

Add support for classification tasks by introducing new classes and methods, refactor benchmarking and task execution logic for better modularity, and enhance model fine-tuning and inference processes with abstract base classes. Include new configuration files and scripts for data preparation and model setup.

New Features:

• Introduce support for classification tasks by adding new classes and methods for handling classification-specific operations.
• Add new configuration files for various model representations and datasets, enabling flexible model training and evaluation setups.

Enhancements:

• Refactor the benchmarking and task running logic to use a more modular and abstract base class structure, improving code organization and reusability.
• Enhance the task runner to dynamically map and execute tasks based on a predefined task map, allowing for more flexible task execution.
• Improve the model fine-tuning and inference processes by introducing abstract base classes and methods, facilitating easier extension and customization.

Documentation:

• Add new configuration files for different model setups, providing examples and templates for various training scenarios.

Chores:

• Add multiple new configuration files and scripts for data preparation and processing, supporting the new classification feature and other model setups.

[sourcery-ai[bot]]

Reviewer's Guide by Sourcery

This pull request introduces support for classification tasks in the existing machine learning pipeline. It includes significant changes to the model architecture, benchmarking process, and data handling. The changes are implemented across multiple files, with major updates to the core functionality in src/mattext/models/benchmark.py, src/mattext/main.py, src/mattext/models/finetune.py, src/mattext/models/predict.py, and src/mattext/models/score.py. New configuration files and data preparation scripts have also been added to support the classification tasks.

File-Level Changes

Files	Changes
src/mattext/models/benchmark.py src/mattext/models/predict.py	Introduced abstract base classes (BaseBenchmark, BaseInference) to support both regression and classification tasks
src/mattext/models/benchmark.py src/mattext/models/predict.py src/mattext/models/finetune.py	Added new classes for classification tasks (MatbenchmarkClassification, InferenceClassification, FinetuneClassificationModel)
src/mattext/main.py	Updated the main execution flow to include classification tasks
src/mattext/models/score.py src/mattext/models/predict.py	Implemented new metrics and evaluation methods for classification tasks
conf/model/classification_example.yaml conf/model/formation_energy.yaml conf/model/llama_8b.yaml	Added new configuration files for various model representations and datasets
revision-scripts/prep_rep.py revision-scripts/text_rep.py revision-scripts/prep_json.py revision-scripts/mp_classification.py revision-scripts/5fold_split.py	Created new scripts for data preparation and processing
conf/benchmark.yaml conf/llm_sft.yaml conf/model/benchmark_example.yaml	Updated existing configuration files to support new tasks and model representations

---

Tips

- Trigger a new Sourcery review by commenting `@sourcery-ai review` on the pull request. - Continue your discussion with Sourcery by replying directly to review comments. - You can change your review settings at any time by accessing your [dashboard](https://app.sourcery.ai): - Enable or disable the Sourcery-generated pull request summary or reviewer's guide; - Change the review language; - You can always [contact us](mailto:support@sourcery.ai) if you have any questions or feedback.

[n0w0f]

@sourcery-ai review Can you check if the roc_auc and metric computation is done correctly in score.py ?

[kjappelbaum]

As you and Sourcery said, there is a lot of duplication, and I'm unsure if we should merge it in the current form.

[kjappelbaum]

@sourcery-ai review Can you check if the roc_auc and metric computation is done correctly in score.py ?

Is there anything particular you have concerns about, @n0w0f ?

[n0w0f]

@sourcery-ai review Can you check if the roc_auc and metric computation is done correctly in score.py ?

Is there anything particular you have concerns about, @n0w0f ?

I wanted to jsut confirm if the metrics are correctly computed.

[n0w0f]

@kjappelbaum I agree. we need not merge this. I did not want to break the codebase hence the duplications I can clean up after the sprint

[kjappelbaum]

i'd also consider moving to something like unsloth for fine-tuning as it is much faster

[kjappelbaum]

Another point we had mentioned on Zulip was to add

• batching
• progress bars

[n0w0f]

@sourcery-ai review

[n0w0f]

Refactored to incorporate the comments. can you @sourcery-ai review