search

dbmi-pitt / DIKB-Micropublication

Micropublication and Open Data Annotation for drug-drug interaction evidence synthesis
Apache License 2.0
7 stars 1 forks source link

Rejected evidence - see queries/use-cases.txt #15

Open samuelrosko opened 9 years ago

samuelrosko commented 9 years ago

The current model does not include where evidence is rejected, there's also no mp predicate for indicating that

timclark commented 9 years ago

Sam - would you like to have a skype call on this issue?

Briefly, having a predicate for “rejection” is one way to do this - but not how I would recommend approaching the problem.

What I would do is one or both of two things. (1) use a qualifier “REJECTED” or "REJECTED BY KB CURATOR"- qualifiers of all kinds are supported in the model. You can also have an “ACCEPTED” tag (2) if you want to track WHY it is rejected (and/or, why its inverse is ACCEPTED, then assert another claim stating why and citing the evidence you find credible.

The authorship of the accepting or rejecting claims is “KB Curator”, who is privileged in terms of what s/he accepts or rejects being the KB point of view at any given time.

You are essentially raising the question of evidence weighting or credibility in an indirect way. REJECTED is a stronger form of challengedBy and ACCEPTED is a stronger form of supportedBy, with implied authorship “KB Curator”. I would not set up new predicates in that way however.

Tim Clark, Ph.D. Assistant Professor of Neurology, Harvard Medical School Director of Informatics, MassGeneral Institute for Neurodegenerative Disease co-Director, Data and Statistics Core, Massachusetts Alzheimer Disease Research Center website: http://mindinformatics.org mobile: +1 617-947-7098 fax: +1 617-213-5418

On Jul 1, 2015, at 5:16 PM, Samuel Rosko notifications@github.com wrote:

The current model does not include where evidence is rejected, there's also no mp predicate for indicating that

— Reply to this email directly or view it on GitHub https://github.com/dbmi-pitt/DIKB-Micropublication/issues/15.

timclark commented 9 years ago

HI All,

This is an important topic for the DIKB because, as a policy, we track all evidence that is rejected. Rejections are based on the explicit inclusion criteria plus the curator's judgement, and a reason for rejection is always required. Rejected evidence can apply to a single claim or a whole "family" of claims that share some commonality. For example, a poorly done in vitro experiment conducted to assess the metabolism of Drug X can be rejected for all "Drug X substrate-of Y" claims.

I could see qualifiers working for this use case with two additional features:

1) the resource that the qualifer points to should be a simple graph that we can use to query out the reason for rejection, who rejected it, and when.

2) Rather than the claim itself, the subject of qualifiedBy would need to be attached to one of MP data, materials, or method resources in the
support or refutation chain for a claim. Otherwise, we would be incorrectly stating that the curator rejects the claim rather than the use of an evidence item as support or refutation for that claim.

To flesh this out a bit more, here is snippet of RDF points to a claim and provides some data (assuming that having EV_PK_DDI_NR_Data as a sub-class of MP:Data is correct). I have added in mp:qualifiedBy predicate pointing to a made up resource that would later be described elsewhere with the reason for rejection etc. Since this is data, maybe the rejection was because of bad statistical methods etc.

<rdf:Description 
rdf:about="http://purl.org/net/nlprepository/spl-ddi-annotation-poc#ddi-spl-annotation-data-24">
     <rdf:type 
rdf:resource="http://dbmi-icode-01.dbmi.pitt.edu/dikb-evidence/DIKB_evidence_ontology_v1.3.owl#EV_PK_DDI_NR_Data"/>
     <mp:supports 
rdf:resource="http://purl.org/net/nlprepository/spl-ddi-annotation-poc#ddi-spl-annotation-claim-25"/>
    *<mp: qualifiedBy 
rdf:resource="http://purl.org/net/nlprepository/spl-ddi-annotation-poc#ddi-rejection-1"/>*
<dikbD2R:increases_auc>1.069</dikbD2R:increases_auc>
   </rdf:Description>

However, when I look at http://www.jbiomedsem.com/content/5/1/28/figure/F5 and at MP n Protege , I see that the range of mp:qualifiedBy is mp:Sentence. I don't see it in the ontology, but it doesn't seem that MP:Sentence can be an RDF resource like described above. If that is true, and the above make sense, do we need a different property for handling rejected evidence?

ty, -R

On 07/01/2015 05:32 PM, Tim Clark wrote:

Sam - would you like to have a skype call on this issue?

Briefly, having a predicate for “rejection” is one way to do this - but not how I would recommend approaching the problem.

What I would do is one or both of two things. (1) use a qualifier “REJECTED” or "REJECTED BY KB CURATOR"- qualifiers of all kinds are supported in the model. You can also have an “ACCEPTED” tag (2) if you want to track WHY it is rejected (and/or, why its inverse is ACCEPTED, then assert another claim stating why and citing the evidence you find credible.

The authorship of the accepting or rejecting claims is “KB Curator”, who is privileged in terms of what s/he accepts or rejects being the KB point of view at any given time.

You are essentially raising the question of evidence weighting or credibility in an indirect way. REJECTED is a stronger form of challengedBy and ACCEPTED is a stronger form of supportedBy, with implied authorship “KB Curator”. I would not set up new predicates in that way however.

Tim Clark, Ph.D. Assistant Professor of Neurology, Harvard Medical School Director of Informatics, MassGeneral Institute for Neurodegenerative Disease co-Director, Data and Statistics Core, Massachusetts Alzheimer Disease Research Center website: http://mindinformatics.org mobile: +1 617-947-7098 fax: +1 617-213-5418

On Jul 1, 2015, at 5:16 PM, Samuel Rosko <notifications@github.com mailto:notifications@github.com> wrote:

The current model does not include where evidence is rejected, there's also no mp predicate for indicating that

— Reply to this email directly or view it on GitHub https://github.com/dbmi-pitt/DIKB-Micropublication/issues/15.

Richard D Boyce, PhD Assistant Professor of Biomedical Informatics Faculty, Center for Pharmaceutical Policy and Prescribing Faculty, Geriatric Pharmaceutical Outcomes and Gero-Informatics Research and Training Program University of Pittsburgh rdb20@pitt.edu Office: 412-648-9219 Twitter: @bhaapgh

jodischneider commented 8 years ago

One pending question that we have is how to deal with rejected evidence. We've been discussing using mp:Qualifer but it is an mp:Sentence whereas we need a graph.

So far the questions I have are:

General situation:

So far, we've been looking at using mp:Qualifier to track rejection. The main problem seems to be that mp:Qualifier is a mp:Sentence (e.g. declarative statement). While we could make a sentence out of this, "Curator A rejected this for reason B at YYYY-MM-DDTHH:MM:SS-04:00" we need to record all that information separately (e.g. in a graph) so that it can be queried.