Missing data

[liuh236]

Hello! Thanks for your great job!

I find that the data folder is missing. If possible, can you release the dataset or the preprocessing script?

Thanks all.

[ezosa]

Hi! Thank you for your message. We've added the Wikipedia article titles and image urls in our train and test sets in the data directory. We will upload the preprocessing scripts soon.

[liuh236]

thanks a lot!

[liuh236]

btw, which folder will the preprocessing scripts be placed in? ^^

[ezosa]

The utils folder

[liuh236]

Hi! Thank you for your message. We've added the Wikipedia article titles and image urls in our train and test sets in the data directory. We will upload the preprocessing scripts soon.

• Hi! I am curious about how to extract the target text from the original Wikipedia Comparable Corpora. When will the preprocessing script be released? thanks again!

[ezosa]

Hi, We will upload it tomorrow.

[liuh236]

Hi,

There is an error "tag mismatch" while using python extract titles for aligned multilingual Wikipedia articles (https://linguatools.org/tools/corpora/wikipedia-monolingual-corpora/). For example,

tree = ET.parse("wikicomp-2014_deen.xml") xml.parsers.expat.ExpatError: mismatched tag: line 58, column 31

How do you fix it?

[ezosa]

To align titles for multilingual articles, I extract titles from the crosslanguage link in the monolingual corpus, e.g. dewiki-20180920-corpus.xml Monolingual corpora can be found in https://linguatools.org/tools/corpora/wikipedia-monolingual-corpora/

[liuh236]

Thanks for your response. I think there is something wrong with 2014 version (outdated) in the monolingual corpus and the 2018-version is OK. I will take a try.