search

IndoNLP / nusa-crowd

A collaborative project to collect datasets in Indonesian languages.
Apache License 2.0
261 stars 61 forks source link

Configure github action workflow to run nusantara test #249

Closed christianwbsn closed 2 years ago

christianwbsn commented 2 years ago

PoC here: https://github.com/christianwbsn/nusa-crowd/pull/4

For easier testing, let's abuse utilize GitHub actions computing power, and memory How to use: Screen Shot 2022-09-09 at 16 43 15

bryanwilie commented 2 years ago

This looks good, but could the testing also be triggered when someone submitted a PR (not a draft, per se)?

I'm just asking, since I think trying this functionality alone is already ok for a mainstream submission. I think if someone wants to run a test on a different subset, this workflow still need to be enhanced (I just look up at your example here, where the test returns an error due to the subset requirement)

christianwbsn commented 2 years ago

This looks good, but could the testing also be triggered when someone submitted a PR (not a draft, per se)?

I'm just asking, since I think trying this functionality alone is already ok for a mainstream submission. I think if someone wants to run a test on a different subset, this workflow still need to be enhanced (I just look up at your example here, where the test returns an error due to the subset requirement)

Ah, for that to work, I think we need to standardize how we write titles in PR / use branches (people are not really consistent with this). When someone opens a PR, we can't precisely know which data loader needs to be tested, hence we still need to type the dataset name manually. Dataset with subset gives another reason that we still need to trigger it manually rather than automatically trigger it whenever someone submits a PR. Open for new idea for this issue, I think it could be great if we can do that automatically

And for subset, I've already updated it! https://github.com/christianwbsn/nusa-crowd/actions/runs/3035492504