Extract fact-checks from Web Data Commons

[infojunkie]

Tell us about your request Web Data Commons extracts metadata from Common Crawl. We can extract Schema.org ClaimReview entries from this dataset and populate our Fetch database with such entries.

Implementation details

• Call the parsing provider class WebDataCommons or similar
• The tricky part of ClaimReview is understanding which claim is being reviewed. This is primarily expressed in the claimReviewed text attribute and itemReviewed object, which may be empty in some entries. What to do in those cases?
• Assume that the Web Data Commons dataset is addressable via a URI (your choice of file:, http:, etc.)
• Check out the Web Data Commons forum for existing approaches to extract data from this dataset.

[DGaffney]

So it sounds like essentially, Web Data Commons is an aggregator of Schema.org-compliant content, some subset of which is bound to be ClaimReview objects. In principle, adding their ClaimReview objects into the database is a no-brainer then. Should not be too difficult to add this into fetch.