Build ContentMine-based workflow for "main subject" of papers in Wikidata

[Daniel-Mietchen]

ContentMine can analyze papers in various ways, including as to what the most salient terms are, e.g. via https://en.wikipedia.org/wiki/Tf%E2%80%93idf .

It would be nice to harvest that to annotate Wikidata items about papers with the property P921 "main subject".

[Daniel-Mietchen]

As a starting point, it would make sense to go for papers that are already on Wikidata and have a P932 (PMCID) statement.

The query for that is

SELECT ?item ?pmcid WHERE {
  ?item wdt:P31 wd:Q13442814;
        wdt:P932 ?pmcid.  
}
#LIMIT 100

Without the LIMIT command, this just took 6s and gave 334628 results, which sounds like a good maximal size for a test set.

[petermr]

Daniel-Mietchen and I discussed this with the possible outcomes of:

• being hacked at OpenData day
• being incorporated into the WikiFactMine project https://meta.wikimedia.org/wiki/Grants:Project/ContentMine/WikiFactMine

High-level strategy

Collect a corpus of Open articles and carry out supervised term analysis of the content, supported by #wikidata-enhanced dictionaries. Articles with a "main topic" which maps onto #Wikidata items (Q\d+) are likely to have many mentions of the main topic. For example article http://europepmc.org/articles/PMC2491585 mentions

• DENV (Q476209) x 88 // Dengue virus
• Dengue (Q30953) x 8 // Dengue fever
• Yellow fever x 3

and the most common terms (Bag of words) are:

• HLA (Q911125) x 39 // Human leukocyte antigen
• peptide (Q172847) x 33

We can infer that the main topic of the article is Dengue Virus and antigenicity. This is consistent with the title:

Conservation and variability of dengue virus proteins: implications for vaccine design.

The term "vaccine" occurs 16 times in the main text (whereas "HLA" and "peptide" - the mechanism of vaccination is emphasised.

Corpus of articles:

[Daniel-Mietchen]

OK, I've added these to https://www.wikidata.org/wiki/Q24288762#P921 .

How can we scale that up? Can you provide a list of the following kind?

• Wikidata item for scientific article that has a PMCID (see my comment above)
• Wikidata items for the top 3 - 5 topics identified as "main subject" (in the sense of P921) as per your comment above

[Daniel-Mietchen]

Reopening this, as we are still working on it.

[Daniel-Mietchen]

A side project could be to identify the main subject(s) for journals — currently, ca. 40k instances of scientific journal do not have any main subject set in Wikidata Query:

SELECT ?item ?itemLabel WHERE {
  ?item wdt:P31 wd:Q5633421.
  MINUS {?item wdt:P921 ?mainsubject.}
  SERVICE wikibase:label { bd:serviceParam wikibase:language "en". }

}

[Daniel-Mietchen]

As per https://www.wikidata.org/wiki/Property_talk:P921 , P921 is for the "primary topic of a work", which should have a Wikidata entry. It doesn't have to be just one - for instance, the article Causal or not: applying the Bradford Hill aspects of evidence to the association between Zika virus and microcephaly (Q24261170) currently has Zika virus, microcephaly and Bradford Hill criteria.

On Sun, Mar 19, 2017 at 4:41 PM, Stefan Kasberger notifications@github.com wrote:

What do you exactly mean with the term "main subject"?