NCATSTranslator / Feedback

A repo for tracking gaps in Translator data and finding ways to fill them.
7 stars 0 forks source link

Weird ordering/ranking of results for What treats GERD #772

Open khanspers opened 2 months ago

khanspers commented 2 months ago

What drugs may treat gastroesophageal reflux disease (GERD)? (matched on acid reflux)

https://ui.transltr.io/main/results?l=Gastroesophageal%20Reflux%20Disease&i=MONDO:0007186&t=0&r=0&q=64ded059-a50b-41c2-8d85-144a3c85108d

The ordering of results on the first page is weird: The top results is Gefitinib, a tyrosine kinase inhibitor chemotherapy agent. There is one inferred path (from Unsecret agent). The two supporting paths have ~80 publications together (link between Gefitinib and EGFR and link between EGFR and GERD), so maybe that's why it gets listed first, but none of these publications support a direct treats path?

The third result is also a chemotherapy agent, Panitumumab, with one inferred path and 5 clinical trials as evidence. Clinical trials are relevant to the use of Panitumumab in various cancers of the stomach it seems, nothing about GERD.

In contrast, Omeprazole (Known treatment, aka Prilosec/Losec) is listed as the 7th result, with 3 direct paths supported by 181 publications and 23 clinical trials.

Why is Gefitinib and Panitumumab listed above Omeprazole with weaker evidence and no direct paths? Several known treatments on page 2 and 3 also have more evidence, for example Ranitidine, Lansoprazole, Histamin H2 Antagonists.

sierra-moxon commented 1 month ago

Hi @sharatisrani - can this example be added to your tests w/re to ordering results?

sierra-moxon commented 1 month ago

from TAQA: in the last weeks, some work has been done by Improving to fix the scores related to evidence, this may be better in TEST soon.

Andy's relay session is to explain "why" this happens, not fix it. Might get a pass on the issue if the user can reason about why this is scored high vs. low, etc.

action:

52 on O&O is related.

jh111 commented 1 month ago

Added QA pairs. Added @maximusunc to determine which normalized OutputID should be used.

sstemann commented 4 hours ago

this still occurs (although now Omeprazole is Esomeprazole)

https://ui.test.transltr.io/main/results?l=Gastroesophageal%20Reflux%20Disease&i=MONDO:0007186&t=0&r=0&q=bdbb6b2f-ee7d-4447-8c7a-34d95afea226

the ordering is controlled by the Rank, so while Sugeno is 1 (scale is 0-1, not 5), the rank is 15. I think this is an O&O ticket.

image

The ARA scores are (scale is 0-1): ARAX - .82 Unsecret - .26 Improving - .61 BTE - 1 ARAGORN - 9.1

image

O&O Flattened Excel gastroesophageal reflux disease_test_2024_7-5_17_28_bdbb6b2f-ee7d-4447-8c7a-34d95afea226.xlsx