Write biblio details import lib

dosumis commented 3 years ago

(Ticket may not belong here)

We need a generic biblio details import extension for VFB pipeline2. This should use lookups via NCBI pubmed API or DOI to pull general bibliographic information into the integration layer (triplestore).

To investigate - can we follow/re-use some existing standard for this?

dosumis commented 3 years ago

NCBI pubMed API doc https://www.ncbi.nlm.nih.gov/books/NBK25497/ https://www.doi.org/factsheets/DOIProxy.html

dosumis commented 3 years ago

Also try EuroPMC - might give preprints too

hkir-dev commented 3 years ago

Coverage of the EuroPMC seems good: https://www.ebi.ac.uk/europepmc/webservices/rest/search?query=DOI%3A10.1101%2F2020.02.29.970558&resultType=lite&cursorMark=*&pageSize=25&format=dc

We can re-use Dublin Core response format of this service.

dosumis commented 3 years ago

Looks great. I like the idea of following DC response format as standard. Assume importing to triple-store => individual with IRI http://europepmc.org/abstract/PPR/PPR115538 ?

We need to think through how to wire these up to Classes/Individuals - or if we don't, how to index so details can easily be retrieved by indexed ID.

References to publications are commonly found in IRIs on definitions, e.g. see PMID here:

"definition": [ "{"annotations":{"database_cross_reference":["PMID:12162730","MESH:A08.637.200","GOC:dsd","MESH:D001253","GOC:tfm","PMID:11746784","PMID:20942978","http://en.wikipedia.org/wiki/Astrocyte","PMID:12898703"]},"value":"A class of large neuroglial (macroglial) cells in the central nervous system - the largest and most numerous neuroglial cells in the brain and spinal cord. Astrocytes (from 'star' cells) are irregularly shaped with many long processes, including those with 'end feet' which form the glial (limiting) membrane and directly and indirectly contribute to the blood-brain barrier. They regulate the extracellular ionic and chemical environment, and 'reactive astrocytes' (along with microglia) respond to injury."}" ]

May also be found on edges. Given that - maybe an indexed key lookup would be best?

hkir-dev commented 3 years ago

Our current structure is as follows:

    <owl:Axiom>
        <owl:annotatedSource rdf:resource="http://www.semanticweb.org/brain_data_standards/AllenDendClass_CS202002013_1"/>
        <owl:annotatedProperty rdf:resource="http://purl.obolibrary.org/obo/IAO_0000115"/>
        <owl:annotatedTarget rdf:datatype="http://www.w3.org/2001/XMLSchema#string">A GABAergic interneuron ...</owl:annotatedTarget>
        <oboInOwl:hasDbXref rdf:datatype="http://www.w3.org/2001/XMLSchema#string">DOI:10.1101/2020.02.29.970558</oboInOwl:hasDbXref>
        <oboInOwl:hasDbXref rdf:datatype="http://www.w3.org/2001/XMLSchema#string">DOI:10.1101/2020.03.31.016972</oboInOwl:hasDbXref>
    </owl:Axiom>

I was planning to add one more annotation to this axiom using dc:source :

<dc:source rdf:resource="http://europepmc.org/abstract/PPR/PPR115538/>

Do you have a type recommendation for biblio instances (at worst we can use BibliographicResource)?

In the vfb_dumps we can construct an explicit relation (dc:source?) between AllenDendClasses and the related biblio instances.

dosumis commented 3 years ago

Proposal:

Add rdf as returned by EuroPMC API - dc structure, but add an additional triple with value query string.

e.g.

query with https://www.ebi.ac.uk/europepmc/webservices/rest/search?query=doi:10.1101%2F2020.02.29.970558&resultType=lite&cursorMark=*&pageSize=25&format=dc

Add:

http://europepmc.org/abstract/PPR/PPR115538 :DOI 10.1101/2020.02.29.970558 .

Or PMID.

hkir-dev commented 3 years ago

skos:exactMatch relation used for storing query string:

http://europepmc.org/abstract/PPR/PPR115538 skos:exactMatch "DOI:10.1101/2020.02.29.970558"

Implementation of the related feature is completed and PR created https://github.com/VirtualFlyBrain/vfb-pipeline-collectdata/pull/20

obophenotype / brain_data_standards_ontologies

Write biblio details import lib #135