NCATSTranslator / testing

Materials and tools for testing Translator components
1 stars 9 forks source link

Explain the mechanism of action of Tacrine on Alzheimer's #93

Open sstemann opened 3 years ago

sstemann commented 3 years ago

In this query we tried two versions: Query: tacrine.json PK: 0d2470b1-2a98-46eb-8317-347ed66a589a MONDO: 0004975 Control: We were looking specifically for the acetylcholinesterase activity and how it was being affected.

image

Query: tacrine2.json PK: a03dfea0-581a-422d-9d1a-3625a82c5e5a NCBIGene: 43

image

Results Tracking Sheet

yakaboskic commented 3 years ago

Currently CHP doesn't support biolink:BiologicalProcessOrActivity which is what directly lead to no results. However, we also don't have this disease or chemical substance in our model so we wouldn't have been able to return do to that constraint as well.

vdancik commented 3 years ago

MolePro as KP does not accept multi-hop queries.

suihuang-ISB commented 3 years ago

GENERAL COMMENTS on this use case - notes from our internal discussions

However interesting and useful in exposing broader limitations of the TRANSLATOR, this use case is epistemologically ill-posed. I do not think that getting a meaningful answer to this query ("Explain the mechanism of action of Tacrine on Alzheimer' ") exemplifies a good and typical use of the Translators.

So here are some of the issues that hopefully help in formulating test cases:

[1] The query is not well-formulated: It is a MECHANISM that explain something - a MECHANISM is not that which is being explained. At best, the TRANSLATOR would "come up" with (uncover) the KNOWN or a NEW mechanism that explains the use of TACRINE in Alzheimer. The query is formulated as if the true MECHANISM I already known and the user wants to explain the explanatory mechanism, i.e. go deeper.

[2] Even if correctly phrased (see [1]), finding an explanatory mechanism for TACRINE's potential efficacy in treating Alzheimer's disease (AD) would indeed be a good use of the Translator. Such "explain" scenarios is the goal of Workflow D. However, MECHANISIM of ACTION (MOA) of a drug is a very tricky thing -see [3] and makes it difficult to be used to benchmark a result returned for evaluating a basic functionality of the Translator.

[3] The truth in biomedicine is that most MECHANISMS in pharmacology, which has motivated a drug development OR has been retrospectively revealed, turn out to be incorrect or incomplete. For instance, Gleevec (imatinib) may not just act by blocking c-kit or PDGFR or Bcr-Abl kinases - thereby killing the cell, but it may for instance also affect the supporting tumor stroma (which expresses PDGF) or it may inhibit T_reg (which inhibit anti-tumor T cell immunity). 100s of papers for each drug offer potential mechanism, typically. In brief, molecular/cellular interactions or relationships (which is most of what the Translator is about) is just one layer of explanation. Molecules affect cells, which collectively control tissue states, which in turn impact organ function which mediate organ-organ interactions (eg the gut talks to the brain etc..). This (patho)physiology network is not (yet) represent in the Translator. The Translator network does not explicitly intergrate the various scales or levels of description.

[4] As mentioned, it could be that this query is also epistemologically ill-posed in a more fundamental way: Maybe the user does not seek (A): the "Molecular ground truth" that explains a MOA based on factual knwledge of molecular interactions and functions to connect the dots to an explanatory mechanism not previously recognized but rather they seek (B): the "believed mechanism" that is commonly thought as the accepted common operational knowledge among experts independent of actual veracity. In other words, the user is interested in what hypothesis has been around that have motivated the use of Tacrine in AD.

[5] I would say, referring to [4] the Translator should focus on (A): delivering molecular-mechanistic explanations based on (a chain) established molecular/regulatory interactions. With the type of query (B), we move the intended use of Translator from an integrator of factual knwledge deposited in knowledge sources and predigested by KPs, into some sort of a search engine for historians interested in distilling documented thought processes. This is an entirely different task. Of course (B) is also interesting, in some other way, and would emphasize another layer of sophistication better suited for NLP.

[6] Back to Tacrine: The MOA of Tacrine in the literature is only hypothesized!! This use case thus asks for a hypothesized pathogenetic mechanism of a disease as the basis for the MOA of a drug that treats it. The pathogenetic mechanism here is a hypothesis and is not ground truth but is what had led to the proposal of trying Tacrine to increase Acetylcholine in AD: It was the cholinerg hypothesis of AD of the 1970s: Loss of cholinergic neuronal projections to the frontal lobe (the latter is observed in AD as such - this is a solid fact). -But it is still only a hypothesized causal mechanism of AD, part of the so called neurotransmitter hypothesis, much as the other dozen hypothesis of AD that have stimulated (failed) clinical trials: the virus hypothesis, inflammation hypothesis, neurovascular hypothesis, amyloid hypothesis, Tau hypothesis...

[7] Hence, bcause the porposed MOA of TACRINE is based on an old hypothesis for the genesis of AD, this query belongs to province of (B) - see [5]. I think finding old hypotheses is not the best use of the Translator and this is a task is better done by a historian of science by going to a library. At best, this could be an interesting NLP task per se if electronic records of the old literature exist.

[8] I emphasize 'history" because AchE inhibitors, such as Tacrine not only share the MOA with nerve gases used in WW II but also because in 2012 the French Pharmaco-economic Committee had declared the efficacy of AchE inhibitors to be "low" after decades of inconclusive trials showing low symptomatic relief and very low tolerability.

colleenXu commented 3 years ago

BTE now handles both queries and returns results, after its long-awaited update! @andrewsu I think we can be unassigned from this issue.

(note that the link and screenshot are for TRAPI v1.1)