Adds SAMsum, Q-Msum and SUMMScreen dataset

Yale-LILY / SummerTime

An open-source text summarization toolkit for non-experts. EMNLP'2021 Demo

https://arxiv.org/abs/2108.12738

Apache License 2.0

264 stars 30 forks source link

Adds SAMsum, Q-Msum and SUMMScreen dataset #31

Closed MuroriM closed 3 years ago

MuroriM commented 3 years ago

Add SAMsum dataset:

Adds the relevant changes to huggingface_datasets.py and init.py to download and load SAMsum

Does minor edits to fix datasets naming convention on Scisummnet

Modifies: non_huggingface_datasets.py

MuroriM commented 3 years ago

I've fixed how the dialogue instances are stored. Each source instance is now a list of strings in the form of '["speaker1 : utter..", "speaker2 : utter..."]

niansong1996 commented 3 years ago

LGTM. @MuroriM Can you confirm that the tests are passing?

MuroriM commented 3 years ago

It's passing the tests. I also modified to check for list[str] instances on source if it's a dialogue dataset

niansong1996 commented 3 years ago

Awesome, merging this now