Filter German datasets - Githubissues

dennlinger commented 1 year ago

Re-opening #1 with new task focus, which is first to filter all of the existing German datasets. Also depends on "finishing" a working version of Cleaner.

List of obtained datasets:

[x] Klexikon (own version and on Huggingface)
[x] MLSUM (available on Huggingface)
[x] Wiki_lingua (available on Huggingface)
[x] MassiveSumm (got access via the author to the German subset)
[x] LegalSum (repository available, but access format unsure, because they internally use a database?)
[x] Own source 2 (access available, but no dedicated German subset parsed (yet))

Editing this to reflect each of those points will get a separate issue.

[x] #59
[x] MLSUM. Already linked to the post by Philip May before, but it is really essentially to remove duplicates. Also unsure how to deal with changes to the test set (especially the fully extractive samples strongly skew results in favor if lead-based methods).
[x] Wiki_lingua. Still have to look at the data samples, because the paragraph vs full text bothers me, mainly due to the total length / instructional type of samples.
[x] MassiveSumm. Preliminary experiments in #48 show that there are some issues with duplicate content.
[x] LegalSum. Have no access yet, so not more information available than author-reported scores.
[x] Own source 2. Similar to LegalSum, but some more details known.

dennlinger commented 1 year ago

Also blocker by #52, given that we want to preserve previous splits.

dennlinger commented 1 year ago

Opened issue for LegalSum: https://github.com/sebimo/LegalSum/issues/1
Currently awaiting response from the authors, but it seems we can easily use this, since the downloaded files already look promisingly clean. Unclear which columns to use, and also some discrepancy in the provided file count.

dennlinger commented 1 year ago

Also #52 done now, so cleaning is generally possible with the preliminary filters.

On top, though, it would be great to compare the stats before and after filtering. Need to include Jiahui's code first for this, see #50.

dennlinger commented 1 year ago

Incorporated in #63 .

dennlinger / summaries

Filter German datasets #49