Add Multilingual Datasets

Yale-LILY / SummerTime

An open-source text summarization toolkit for non-experts. EMNLP'2021 Demo

https://arxiv.org/abs/2108.12738

Apache License 2.0

264 stars 30 forks source link

Add Multilingual Datasets #112

Closed haileyschoelkopf closed 2 years ago

haileyschoelkopf commented 2 years ago

We currently have only 1 multilingual dataset with 5 languages, but have multilingual models that theoretically support many more than that. I would like to implement:

XLSum (45 languages) which is already in Huggingface
MassiveSumm (92 languages) which is not in Huggingface

So that we have a much greater spread of multilingual data to test on.

niansong1996 commented 2 years ago

@NickSchoelkopf Is this done? Seems like we can close this?

haileyschoelkopf commented 2 years ago

@niansong1996 Yes, this can be closed!