Fix - Adding annotations from Tables & Lists - Githubissues

idio / json-wikipedia

Json Wikipedia, contains code to convert the Wikipedia xml dump into a json dump. Questions? https://gitter.im/idio-opensource/Lobby

17 stars 2 forks source link

Fix - Adding annotations from Tables & Lists #8

Closed dav009 closed 9 years ago

dav009 commented 9 years ago

Many annotations take place in tables like in [1] or Lists like in [2].

This PR considers each row of the table as a paragraphWithLinks.
This PR considers each item in the list as a paragraphWithLinks. . (this includes the section : See also at the bottom of articles)
In both cases we extract the text with correct anchors (matching spans in text)

while doing this spotted: https://github.com/idio/json-wikipedia/issues/7 so two of the links in [2] wont be caught by this PR.

[1] https://en.wikipedia.org/wiki/International_Military_Tribunal_for_the_Far_East [2] https://en.wikipedia.org/wiki/Hayami

keynmol commented 9 years ago

Would be good to gauge the increase in annotations this gives - two runs on a small to see how the size of sfAndTotalCounts increases