NCATSTranslator / Feedback

A repo for tracking gaps in Translator data and finding ways to fill them.
7 stars 0 forks source link

Weak predicate patterns supported for Lactose Intolerance #601

Closed TranslatorIssueCreator closed 1 month ago

TranslatorIssueCreator commented 11 months ago

Type: Bug Report

URL: https://ui.test.transltr.io/main?fm=true

ARS PK: faf05624-2424-4152-8b68-badacbb03f43

Steps to reproduce:

select pesticides as ChEBI role (filter option)

Screenshots:

sandrine-muller-research commented 11 months ago

Ergocalciferol Mebendazole Bithionol Thiabendazole Levamisole Methamidophos Potassium bicarbonate Triclosan

They are all pesticides (not likely to treat a disease!), and their predicates chain pattern is: ... --> treats --> phenotype of --> ... That does not seem a very convincing reasoning (although ranked pretty high), particularly when it seems to treat a lot of diseases (and only that):

image

Retruned by ARAGORN and BTE. Adding this as test assets

gglusman commented 11 months ago

Egocalciferol is indeed used as a rat poison, but also as a synthetic vitamin D... it's not just a pesticide. A drug being labeled a 'pesticide' doesn't immediately make it a wrong answer to every question. I'd say that the issue here is the somewhat weak reasoning pattern of treating symptoms... though that is not infrequently the only thing left to do.

sandrine-muller-research commented 11 months ago

related to feedback from @khanspers on tacrolimus, same predicate pattern for Tacrolimus:

image
sandrine-muller-research commented 11 months ago

yes, fully agree with you @gglusman ! I use the ChEBI role "pesticides" as a surrogate to test those week patterns. I realized that in many queries it comes back and has weak predicates patterns. Usually I only reports issues when the pattern is weak if I have a doubt either I have time to search the literature or I do not report it. I'll change the title though, good point.

sierra-moxon commented 4 months ago

for this one, I hope the review team can come up with a decision on a few key things in this issue: 1) is the reasoning path reasonable? 2) is the result of that reasoning path scored appropriately in the context of the evidence? 3) is this ticket a priority to handle and are the people assigned the right ones?

sierra-moxon commented 4 months ago

from TAQA:

Are these bad answers because the reasoning is bad, or just bad answers in general. If they are just bad, we put in a test to fix that. If the reasoning is bad, (they are ok answers, but some are uncomfortable with the reasoning paths, then this is a discussion topic and/or handle disagreements of what is good reasoning).

sandrine-muller-research commented 2 months ago

@sierra-moxon Like @gglusman pointed out, the compounds might not be bad per se. As a user, when I see that:

The association of all those 3 points makes me say it does not seem a good answer at all. I think it should be a question for the scoring session/team.

sandrine-muller-research commented 1 month ago

retested today. None of the pesticides I found earlier show up now! (Ergocalciferol,Mebendazole, Bithionol,Thiabendazole,Levamisole, Methamidophos, Potassium bicarbonate, Triclosan Closing