shunk031 / huggingface-datasets_JGLUE

JGLUE: Japanese General Language Understanding Evaluation for huggingface datasets
https://huggingface.co/datasets/shunk031/JGLUE
9 stars 2 forks source link

Specify open encoding #6

Closed polm closed 1 year ago

polm commented 1 year ago

This adds the file encoding to two open calls.

For some reason, in one particular cluster environment I use, not doing this causes errors because the files are opened in ascii mode. The environment should be completely UTF-8 so not sure why it's happening, but this modification fixes it and shouldn't cause issues even in environments where it's not strictly necessary.