Closes #275 #16 #274 | Create dataset loader for INDspeech_TELDIALOG_LVCSR, Liputan6, & INDspeech_NEWS_EthnicSR

[ziweiji]

Please name your PR after the issue it closes. You can use the following line: "Closes #ISSUE-NUMBER" where you replace the ISSUE-NUMBER with the one corresponding to your dataset.

Checkbox

• [x] Confirm that this PR is linked to the dataset issue.
• [x] Create the dataloader script nusantara/nusa_datasets/my_dataset/my_dataset.py (please use only lowercase and underscore for dataset naming).
• [x] Provide values for the _CITATION, _DATASETNAME, _DESCRIPTION, _HOMEPAGE, _LICENSE, _URLs, _SUPPORTED_TASKS, _SOURCE_VERSION, and _NUSANTARA_VERSION variables.
• [x] Implement _info(), _split_generators() and _generate_examples() in dataloader script.
• [x] Make sure that the BUILDER_CONFIGS class attribute is a list with at least one NusantaraConfig for the source schema and one for a nusantara schema.
• [x] Confirm dataloader script works with datasets.load_dataset function.
• [x] Confirm that your dataloader script passes the test suite run with python -m tests.test_nusantara --path=nusantara/nusa_datasets/my_dataset/my_dataset.py.
• [ ] If my dataset is local, I have provided an output of the unit-tests in the PR (please copy paste). This is OPTIONAL for public datasets, as we can test these without access to the data files.

[ziweiji]

python -m tests.test_nusantara nusacrowd/nusa_datasets/indsp_teldialog_lvcsr/indsp_teldialog_lvcsr.py --subset_id indsp_teldialog_lvcsr

[SamuelCahyawijaya]

/test dataset=liputan6

[github-actions[bot]]

Run result

Check test log here: https://github.com/IndoNLP/nusa-crowd/actions/runs/3160588702

[SamuelCahyawijaya]

/test dataset=liputan6 subset_id=liputan6_canonical

[github-actions[bot]]

Run result

Check test log here: https://github.com/IndoNLP/nusa-crowd/actions/runs/3160615466

[SamuelCahyawijaya]

/test dataset=indsp_teldialog_lvcsr

[github-actions[bot]]

Run result

Check test log here: https://github.com/IndoNLP/nusa-crowd/actions/runs/3160655781

[SamuelCahyawijaya]

/test dataset=indsp_teldialog_lvcsr

[github-actions[bot]]

Run result

Check test log here: https://github.com/IndoNLP/nusa-crowd/actions/runs/3160710538

[SamuelCahyawijaya]

The liputan6 & indsp_teldialog_lvcsr datasets looks good btw

[holylovenia]

/test dataset=indsp_news_ethnicsr subset_id=indsp_news_ethnicsr_Jawa_1

[github-actions[bot]]

Run result

Check test log here: https://github.com/IndoNLP/nusa-crowd/actions/runs/3167636054

[gentaiscool]

@ziweiji hi, I was wondering if you can make minor changes requested by @holylovenia It would be great if you can make it. Thanks!

[SamuelCahyawijaya]

/test dataset=indsp_news_ethnicsr subset_id=indsp_news_ethnicsr_jv_overlap

[SamuelCahyawijaya]

/test dataset=indsp_news_ethnicsr subset_id=indsp_news_ethnicsr_jv_nooverlap