search

yahoojapan / JGLUE

JGLUE: Japanese General Language Understanding Evaluation
Creative Commons Attribution Share Alike 4.0 International
305 stars 19 forks source link

huggingface/datasets support? #8

Closed kaisugi closed 1 year ago

kaisugi commented 1 year ago

Recently I found that JSNLI is (unofficially?) converted to huggingface/datasets formats (https://huggingface.co/datasets/shunk031/jsnli ), which seems really helpful.

I wonder if we could make JGLUE compatible with huggingface/datasets in the same manner...

tomohideshibata commented 1 year ago

Thank you for your suggestion. We will consider this.

JSNLI is another NLI dataset. Our NLI dataset is JNLI.

shunk031 commented 1 year ago

Thank you very much for mentioning my work. If there is a certain demand, I can help you to convert JGLUE into the huggingface dataset format.

I'm currently planning to create the dataset format as my personal repository, not in the form of a pull request.

tomohideshibata commented 1 year ago

@shunk031 Thanks! If we have any problems, we may ask you a question.

shunk031 commented 1 year ago

Hi, all. Thank you for releasing the awesome benchmark datasets!

I have implemented the huggingface datasets (HF) for the JGLUE. The code is published in the following repositories: https://github.com/shunk031/huggingface-datasets_JGLUE

If you have any questions, please contact me via the github issue. I'm currently considering reimplementing the baseline model using the HF datasets.