SEACrowd / seacrowd-datahub

A collaborative project to collect datasets in SEA languages, SEA regions, or SEA cultures.
Apache License 2.0
55 stars 54 forks source link

Closes #479 | Add Dataloader Aya Evaluation Suite #669

Closed muhammadravi251001 closed 1 month ago

muhammadravi251001 commented 1 month ago

Title: Add Dataloader Aya Evaluation Suite

First line PR Message: Closes https://github.com/SEACrowd/seacrowd-datahub/issues/479

Test: You can check this dataloader implementation by copy-pasting this command (to make a faster evaluation):

python -m tests.test_seacrowd seacrowd/sea_datasets/aya_evaluation_suite/aya_evaluation_suite.py --subset_id aya_evaluation_suite_ceb
python -m tests.test_seacrowd seacrowd/sea_datasets/aya_evaluation_suite/aya_evaluation_suite.py --subset_id aya_evaluation_suite_tha
python -m tests.test_seacrowd seacrowd/sea_datasets/aya_evaluation_suite/aya_evaluation_suite.py --subset_id aya_evaluation_suite_mya
python -m tests.test_seacrowd seacrowd/sea_datasets/aya_evaluation_suite/aya_evaluation_suite.py --subset_id aya_evaluation_suite_zsm
python -m tests.test_seacrowd seacrowd/sea_datasets/aya_evaluation_suite/aya_evaluation_suite.py --subset_id aya_evaluation_suite_jav
python -m tests.test_seacrowd seacrowd/sea_datasets/aya_evaluation_suite/aya_evaluation_suite.py --subset_id aya_evaluation_suite_ind
python -m tests.test_seacrowd seacrowd/sea_datasets/aya_evaluation_suite/aya_evaluation_suite.py --subset_id aya_evaluation_suite_vie
python -m tests.test_seacrowd seacrowd/sea_datasets/aya_evaluation_suite/aya_evaluation_suite.py --subset_id aya_evaluation_suite_sun
python -m tests.test_seacrowd seacrowd/sea_datasets/aya_evaluation_suite/aya_evaluation_suite.py --subset_id aya_evaluation_suite_ace
python -m tests.test_seacrowd seacrowd/sea_datasets/aya_evaluation_suite/aya_evaluation_suite.py --subset_id aya_evaluation_suite_bjn
python -m tests.test_seacrowd seacrowd/sea_datasets/aya_evaluation_suite/aya_evaluation_suite.py --subset_id aya_evaluation_suite_khm
python -m tests.test_seacrowd seacrowd/sea_datasets/aya_evaluation_suite/aya_evaluation_suite.py --subset_id aya_evaluation_suite_lao
python -m tests.test_seacrowd seacrowd/sea_datasets/aya_evaluation_suite/aya_evaluation_suite.py --subset_id aya_evaluation_suite_min

Checkbox

muhammadravi251001 commented 1 month ago

Thanks @muhammadravi251001 for the dataloader. LGTM!

Thanks for the approval, Ms!