Dev - Githubissues

spencermountain / dumpster-dive

roll a wikipedia dump into mongo

Other

240 stars 46 forks source link

Dev #67

Closed spencermountain closed 5 years ago

spencermountain commented 5 years ago

v4

major json format changes from wtf_wikipedia v6.0.0
get skip_redirects actually working
reduce default batch_size even lower
add verbose_skip option, to log disambig/redirect skipping