Note:
Despite only having 993 data instances, there are almost 5.9M tokens that need to be processed.
the testcase ran successfully, but took my laptop 150s. test case
If anyone knows how I can optimize this, please do tell me.
Checkbox
[X] Confirm that this PR is linked to the dataset issue.
[X] Create the dataloader script seacrowd/sea_datasets/indoler/indoler.py (please use only lowercase and underscore for dataset naming).
[X] Provide values for the _CITATION, _DATASETNAME, _DESCRIPTION, _HOMEPAGE, _LICENSE, _URLs, _SUPPORTED_TASKS, _SOURCE_VERSION, and _SEACROWD_VERSION variables.
[X] Implement _info(), _split_generators() and _generate_examples() in dataloader script.
[X] Make sure that the BUILDER_CONFIGS class attribute is a list with at least one SEACrowdConfig for the source schema and one for a seacrowd schema.
[X] Confirm dataloader script works with datasets.load_dataset function.
[X] Confirm that your dataloader script passes the test suite run with python -m tests.test_seacrowd seacrowd/sea_datasets/indoler/indoler.py.
[ ] If my dataset is local, I have provided an output of the unit-tests in the PR (please copy paste). This is OPTIONAL for public datasets, as we can test these without access to the data files.
Closes #351
Note: Despite only having 993 data instances, there are almost 5.9M tokens that need to be processed. the testcase ran successfully, but took my laptop 150s. test case If anyone knows how I can optimize this, please do tell me.
Checkbox
seacrowd/sea_datasets/indoler/indoler.py
(please use only lowercase and underscore for dataset naming)._CITATION
,_DATASETNAME
,_DESCRIPTION
,_HOMEPAGE
,_LICENSE
,_URLs
,_SUPPORTED_TASKS
,_SOURCE_VERSION
, and_SEACROWD_VERSION
variables._info()
,_split_generators()
and_generate_examples()
in dataloader script.BUILDER_CONFIGS
class attribute is a list with at least oneSEACrowdConfig
for the source schema and one for a seacrowd schema.datasets.load_dataset
function.python -m tests.test_seacrowd seacrowd/sea_datasets/indoler/indoler.py
.