Information Verification Taxonomies

[jmatsushita]

There is (a google group called Information Verification Ecosystem). I've added you to it. We had this discussion about quads #2 with @johngriffin from Atchai when we worked on NewsVerify (Demo site) with Internews.

Just doing a brain dump copy paste in the issue and let's see later how we make these into different documents and different issues.

Uses Cases

These where the long term use cases we were looking at:

• One fact-checker from a single organisation can add his pieces of evidence to an event. They can be viewed and added to by his colleagues before being published.
• Factcheckers from different organisation could choose to publish data only to each other and allow editors to check aggregated fact-checking information before deciding to publish. Organisations could have machine reasoning rules that process publicly available verification meta-data to make decision, or recommend actions to editors.
• Publishers or Bloggers could opt-in by activating their CMSes option to activate a verification workflow and publish the metadata publicly.
• Web platforms could allow the crowd to add their own contribution to the evidence base around an event (a la Andy Carvin or Paul Lewis).
• Web platforms could crawl the web to try and identify web posts about similar events (pivoting on something like Freebase?) and display or reason on verification meta-data.
• We should be able take care of a workflow such as the one I believe Meedan Checkdesk is envisioning with the audience providing new evidence (which could be linked with the generic attribute hasVerificationComponent) which would be added, leaving the choice to the original author to change the verification status of the piece.

  Taxonomies

And these are the design choices we had made at the time on the taxonomy design side of things (from this paragraph):

Verification status

The verification status is an editable taxonomy. It communicates how reliable/trustworthy/verified a piece of news is. The terms in use are:

• verified : the news has been verified to a high standard and can be considered a true and accurate account
• corroborated : the news has been cross-checked and corroborated but there are doubts or outstanding facts
• unverified : the news has not been verified by someone other than the source
• debunked : the news has been revealed as false or misleading

  Verification category

There are three verification categories : where, when and what happened

• Where : where the event happened, GPS coordinates and a text based descriptoin
• When : when the event happened, a single date and time or a to/from date
• What happened : is a description of what happened. This description is also used as the teaser for an event.

The platform may develop to support more verification categories or to visualise the data collected under each verification category.

Source(s)

A source is the same as an author as the content type Author An Author is the authors of original pieces of evidence. Journalism traditionally calls these people 'Sources'. An author is not an authenticated user creating content.

Data Model

Event : something that happened.

• Related Content (pieces of content that are presumably linked to the event, including other events and evidences themselves)
• hasVerificationComponents (a generic relation allowing different organisations to add their own verification components or to deal with unclassified verification components like simple comments)
  • hasSource (the link to the presumed original author of the content)
  • Links to a Source
  • hasEvidence : evidence A, evidence B,...
  • hasDate (the presumed date[s] when the content was authored/captured)
  • Links to a Date
  • hasEvidence
  • hasLocation (the presumed location[s] where the content was authored/captured)
  • Links to a Location
  • hasEvidence
  • hasContext (the presumed actions/protagonists/results/causes/... that are associated with the event)
  • Links to Context (can be various types)
  • hasEvidence

Evidence (some data corroborating or invalidating a particular claim)

• Author (the person submitting the evidence)
• Source (the original source or provider of the evidence, including automated platforms)
• Data (the evidence itself as text, multimedia data, a link,...)

[jmatsushita]

@johngriffin had commented on the data model:

Event

• An event links together a collection of evidence. This evidence will support or deny that something happened, in some place, at some time. These direct properties of the event (what happened, when, where) appear to be inferred (either by machine or by human) from it's linked evidence, by ascribing a truth/trust value to each piece of evidence. Correct?
• It should be possible for each verificationComponont to have multiple members. E.g. some evidence may corroborate everything else, but suggest that the date was 2 years ago. In which case there should be two dates linked from hasDate, with respective evidence linked to each?
• Each organisation can assign it's own truth value to an event itself
  • verified / disproved etc. This leads me to think that the direct properties of an event (what happened, when, where) should be explicit, rather than inferred, so that each organisation's truth value is being assigned to the same assertion - this happened, in this place, at this time.
• Should an event have a verificationComponent called "Source"? Is this not what the author of each piece of evidence is?
• For relating events, perhaps something like SKOS could be used to give some basic meaning to the relationships.
• "Context" - for me this term doesn't really describe what it contains - "the presumed actions/protagonists/results/causes/... that are associated with the event". To me at least, context sounds more like the background circumstances, related but not central to the event.
• Context, in the sense that it is currently used, presents some real problems for exchanging this data and ensuring we're all talking about the same thing. There are established vocabularies to represent things like people (foaf), but the facts of an event could be just about anything, e.g. x said y; z bombed x. To represent these "facts", that each need to be checked and verified, in something like RDF we'd need to invent terms that don't exist in established vocabs. While this is not a problem within a single system, it becomes a problem when data is to be exchanged between systems and organisations.

Semantic Modelling

This is quite a nice simple event ontology that could be used as an alternative to the Freebase one: http://purl.org/NET/c4dm/event.owl# - it's used in the BBC programmes ontology for example. From a publishing point of view, there may be more of an advantage to maintaining compatibility with the schema.org event. Any compatibility only seems possible if we do explicitly state the date and place properties on an event though, at least at time of publishing.

For those who are interested in the semantic modelling, here is a (slightly old, but relevant) paper comparing existing Event ontologies: http://oai.cwi.nl/oai/asset/14783/14783A.pdf

I don't claim to be an expert in semantic modelling, but I've done some before and have sketched out some graphs based on what's currently here, so I could potentially do a first cut when the time comes if there's nobody else in the group.

I appreciate I've dived into technicalities here, so please feel free to discuss more generally the requirements first.