Closed exfalsoquodlibet closed 1 year ago
Added python module to do split of samples to train and dev set stratified by entity categories. I'll add tests when back.
To use it, from the terminal:
python src/utils/stratify_train_test_split_entities.py "path/to/dataset_to_split.jsonl" "path/to/folder/where/to/save/outputs" 0.2
This pull/merge request meets the following requirements:
docs/aqa/aqa_plan.md
docs/aqa/data_log.md
docs/aqa/assumptions_caveats.md
docs
Comments have been added below around the incomplete checks.
Summary
Added python module to do split of samples to train and dev set stratified by entity categories. I'll add tests when back.
To use it, from the terminal:
Checklists
This pull/merge request meets the following requirements:
docs/aqa/aqa_plan.md
)docs/aqa/data_log.md
), if necessarydocs/aqa/assumptions_caveats.md
), if necessarydocs
folderComments have been added below around the incomplete checks.