SEACrowd / seacrowd-datahub

A collaborative project to collect datasets in SEA languages, SEA regions, or SEA cultures.
Apache License 2.0
55 stars 54 forks source link

Closes #615 | Add Dataloader IDK-MRC-NLI #631

Closed muhammadravi251001 closed 1 month ago

muhammadravi251001 commented 3 months ago

Title: Add Dataloader IDK-MRC-NLI

First line PR Message: Closes https://github.com/SEACrowd/seacrowd-datahub/issues/615

Notes

Checkbox

muhammadravi251001 commented 2 months ago

@muhammadravi251001 : Thank you for the update! LGTM!

Thanks for the approval, Sir!

holylovenia commented 2 months ago

A friendly reminder for @luckysusanto to review.

muhammadravi251001 commented 1 month ago

The code works well, but, I noticed that there are only two labels in the dataset: 0 and 2.

I checked the original homepage, and the owner did state that there are 3 labels: Entailment (0), neutral (1), and contradiction (2). However, the original dataset only contains two labels: either entailment or contradiction.

I think it would be better for us to turn "contradiction" into (1) [changed from (2)], and then put a comment/note on the file. I fear that currently, it might cause some confusion for users later on

cc: @holylovenia

It was done on purpose, Lucky. I've already made the explanation/clarification on this comment for the same task of my NLI dataset: https://github.com/SEACrowd/seacrowd-datahub/pull/633#issuecomment-2088094588

luckysusanto commented 1 month ago

I see, in that case, approved!

muhammadravi251001 commented 1 month ago

I see, in that case, approved!

Alright, thanks for the approval, Lucky!