Open gglusman opened 8 months ago
cbizon
commented
8 months ago The first X paths have no support of any kind. They are direct treats results with no supporting publications. They also list as their primary knowledge source either Unsecret agent (which is not a primary knowledge source) or Drug Repurposing Hub, which may be such a source, but the linked page does not clarify how its assertions are made or their strength.
cbizon
commented
8 months ago Also - cyclophosphamide shows up with 0 score in the UI. It appears to only be coming back from ARAX with a score of 0.81. So it seems as though something funky is happening in score normalization / aggregation.
gglusman
commented
7 months ago 
sierra-moxon
commented
7 months ago from TAQA:
we can now see that there are icons to show "text mined" vs. "curated"
we are currently using an older ARS score, that is improving
we see that we have scores on the first page now, which is improving.
users want to understand why the score is 5 vs. 0.27 (maybe more metadata is used to create the score that can show up on the page). will be hard to pull off though. (predicated edge, two paths going into it, some arbitrary amount of curation -> algorithm -> out pops a number). This will be a long process. It's not easy to fix in a month.
clicking on the link needs to go to page 1, not page 6. (it's confusing to see zero and assume its page 6).
the original point might be moot; we need to re-test. (retested PK: https://ui.test.transltr.io/main/results?l=Aggressive%20Systemic%20Mastocytosis&i=MONDO:0020333&t=0&r=0&q=87ded114-fd03-4b31-98a9-a8350f71887f for updated link)
larger issue is novelty. lots of development of scores w/re to novelty. new drugs are recently approved, not showing up any more. only included via text mining.
need to look at novelty and text mining is critically important here.
good case for "treats" predication in phase 2 trial.
inhibitor. AVAP - Sui will look up. @bill-baumgartner to flag for drug name.
ACTION:
the rest of this needs to stay open for review and progress over time.
suihuang-ISB
commented
7 months ago This use case highlights several issues for MVP1 that could be instructive.
FIRST SOME BASICS (i) The nominal search term for this MVP1 is the disease “Aggressive Systemic Mastocytosis (ASM)” - this poses a minor problem because recently WHO has changed the classification. ASM is now one form of the overarching group of ADVANCED SYSTEMIC MASTOCYTOSIS (AdvSM), that is differentiated from the other form: non-advanced SM, of which a major form is INDOLENT SM (which will be relevant for discussion below) . Clinical trials for the former nowadays refer to AdvSM rater than ASM. (ii) In 95% of systemic mastocytosis there is an activating mutation in the protein CKIT - the well established target of the class of ckit kinase inhibitors, of which the most famous is imatinib, long approved for several cancers. Thus, cKit inhibitors are plausible, if not obvious candidates for repurposing based on a the molecular MoA. (iii) One particular cKit inhibitor, avapritinib (AyvakitTM) has been approved by FDA for AdvSM in 2021. Given the clear mechanistic rationale, and safety profile of other cKit inhibitors and the unmet medical need, the FDA application received the breakthrough designation and orphan drug designation and underwent priority review. The drug was FDA approved after a small open-label single-arm Phase 1, 2 study (EXPLORER (NCT02561988) and PATHFINDER (NCT03580655). Thus, KPs an ARA that “infer” approval status based on Phase of trial (typically, Phase 3,4) would have missed this drug!! Please note that the “Phase x” is in principle orthogonal to approval status (yes/no) and more of a guide for study design. There is no “Phase of approval” – as I have heard a lot in the calls. Phase is not an approval status, but part of the IND application (Investigational New Drug request for authorization to administer a drug to humans, starting from Phase 1) based on preclinical data...
ON THE RESULTS and CHALLENGES ChEMBL lists for avapritinib (CHEMBL4204794) Systemic Mastocytosis as indication, pointing to a phase 3 clinical trial. But that is for the INDOLENT form of systemic mastocytosis, and this indication was approved after a larger Phase 3 study, in May 2023 – following an application filed by the manufacturer for indication extension (greed-driven! Bigger market! – required a Phase 3, larger study). ChEMBL however does not distinguish between aggressive and indolent in its indication list! Thus, returning ‘avapritinib’ as treatment in response to the query “What drug may treat AGGRESSIVE systemic mastocytosis” (=Advanced SM) based on the Phase 3 study would be technically inaccurate. (( imPROVING agent only finds OTHER (older) CKIT inhibitors on this MVP1 question – drugs that have long been “dangled” as treatment for ASM – because the NCATS-SPOKE version it uses is from 2021 (imatinib, masitinib), based on DrugCentral or clinical trials for these older drugs. Conversely, while avapritinib is in SPOKE, it is only connected to gastric stromal tumors, the classical indication for Ckit inhibitors, that is also listed in ChEMBL/DrugCentral. ))
But the single correct answer for the “treatment of choice” (=the info that clinicians want) that is FDA approved for ASM/AdvSM would have been: avapritinib.
CONCLUSION:
This case highlights the nuances in getting the correct answer for latest findings for MVP1. It points to the problem with “inference” based on clinical trials, and reveals the challenge of capturing the latest FDA approvals. Textmining, ideally from not only literature, but also from company press releases and FDA approval letter seems to be most reliable way for not missing these latest approvals. This is related to the criteria for NOVELTY. (Why infer if you can do a clever look-up ? – another issue to discuss)
Perhaps one may use this case also to distinguish between two types of ‘inference’ that we have oft lumped together: (A) “hermeneneutic” inference = based on interpretation of written text - and (B) scientific inference = based on biomedical reasoning, wherein existing texts/databases only serve to provide the facts for the mechanistic reasoning. Here the case is clear for (B) as the auxiliary graphs show: CKIt is associated with systemic mastocytosis, therefore, cKIT inhibitors could be repurposed. Great that we got that right! But such an answer would not reflect the “official” and latest treatment of choice (avapritinib) if the user is looking for that. Which one should we rank higher??
sstemann
commented
1 month ago @sierra-moxon in your checklist, i can't figure out if these are OBE?
Making a separate ticket for the scoring aspect of this. https://ui.test.transltr.io/main/results?l=Aggressive+Systemic+Mastocytosis&i=MONDO%3A0020333&t=0&r=623e2d02&q=246f5c17-fd36-4980-92fe-e8cf105e53fc
Top results have barely any support (just one ARA). Several pages in, results start having some publications supporting. The last several pages of results show score 0.00, despite having many supporting publications.